Ai4Users - Qwen 3.6 35B NVFP4 Instruct

Benchmark items
Tool Calling - Irrelevance: Architecture & Artwork 8 passed1 failed	88.9%	IN38 267 OUT1 867
✓	NoToolForApartmentRentCostBFCL/irrelevance_158: apartment rental cost asked; only a new-construction cost estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 463 OUT43
×	NoToolForBestJazzSingerBFCL/irrelevance_164: best Jazz singer asked; only a top-instrument-players search (singers are not instrument players) is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2	Failed	IN10 574 OUT502
✓	NoToolForCathedralCeilingDesignBFCL/irrelevance_157: how to design a cathedral ceiling asked; only a lookup of facts about an existing building is available (retrieves data, does not advise on design) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 459 OUT404
✓	NoToolForHolyBookDefinitionBFCL/irrelevance_144: definition of a holy book asked; only a search-within-a-named-holy-book tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 457 OUT103
✓	NoToolForLectureTeacherBFCL/irrelevance_167: teacher of upcoming lectures asked; only an upcoming-concert details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 460 OUT61
✓	NoToolForMostFamousSculptureBFCL/irrelevance_153: most famous sculpture asked; only a sculpting-tools lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 465 OUT176
✓	NoToolForPianoMaintenanceTipsBFCL/irrelevance_166: piano maintenance tips asked; only an instrument rental-prices lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 460 OUT424
✓	NoToolForScreamArtistBFCL/irrelevance_159: artist behind 'The Scream' asked. NOTE: the only tool (artwork_search) requires a museum_location that the user did not provide, so it cannot be correctly called; BFCL labels this irrelevant. Kept faithfully with ground truth = no tool call (flagged as borderline).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 464 OUT18
✓	NoToolForSculptureRockTypeBFCL/irrelevance_152: which rock suits a garden sculpture asked; only a sculpture 3D-model creator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 465 OUT136
Tool Calling - Irrelevance: Buildings, Sculpture & Music 7 passed1 skipped	100.0%	IN23 003 OUT1 377
–	NoToolForCelloInstrumentTypeBFCL/irrelevance_165: instrument classification of a cello asked. NOTE: BFCL labels this irrelevant, but the only tool (get_instrument_info with detail=type) arguably answers it; flagged for review as a borderline BFCL item. Kept faithfully with ground truth = no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_instrument_info' with detail='type' answers 'what type of instrument is a cello' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForDeclarationSignersBFCL/irrelevance_137: who signed the declaration asked; only a historical-event DATE lookup is available (returns a date, not the signatories) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 284 OUT281
✓	NoToolForEiffelTowerHeightBFCL/irrelevance_156: Eiffel Tower height asked; only an architecture-plan generator (creates a new design, does not retrieve facts) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 288 OUT88
✓	NoToolForFamousDancersBFCL/irrelevance_162: famous dancers asked; only a museum-visual-artists lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 290 OUT232
✓	NoToolForReformationInitiatorBFCL/irrelevance_145: who initiated the Reformation asked; only a religious-event YEAR lookup is available (returns a year, not a person) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 283 OUT81
✓	NoToolForScientistDefinitionBFCL/irrelevance_143: definition of "scientist" asked; only a named-historical-figure info lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 280 OUT251
✓	NoToolForSellingGuitarBFCL/irrelevance_163: how to sell a guitar asked; only an instrument-tuning tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 285 OUT301
✓	NoToolForSimilarBlueColorBFCL/irrelevance_148: a color similar to blue asked; only a complementary-color tool (opposite, not similar) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 293 OUT143
Tool Calling - Irrelevance: Cell Biology & Genetics 8 passed2 skipped	100.0%	IN30 331 OUT2 474
✓	NoToolForClottingBloodCellsBFCL/irrelevance_32: blood cell type responsible for clotting asked; only a nucleus-count-based cell-type classifier is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 792 OUT149
✓	NoToolForCommonAncestorBFCL/irrelevance_41: common ancestor of two animals asked (phylogenetics knowledge); only a fibonacci-sequence calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 791 OUT177
✓	NoToolForEvolutionaryAlgorithmInGameAIBFCL/irrelevance_43: how to apply an Evolutionary Algorithm in game AI asked (advice); only a creature-evolution simulator is available (wrong context).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 792 OUT684
✓	NoToolForGeneVariantFrequencyBFCL/irrelevance_36: gene-variant frequency in a population asked; only a dominant-trait-from-alleles tool is available (the true-match variant-frequency tool is deliberately isolated in A4).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 797 OUT220
–	NoToolForIdentifyingGeneticSequenceBFCL/irrelevance_33: interpret a genetic code sequence asked; only a species-identification-from-sequence tool is available (returns a species, not what the sequence means).0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'identify_species' (identifies a species from a genetic sequence) plausibly answers the query about a DNA sequence, so a competent model may call it - BFCL-mislabeled/borderline-relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForIncreasingDeerPopulationBFCL/irrelevance_39: how-to advice on growing an animal population asked; only a population-growth CALCULATOR (needs numeric rates) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 791 OUT300
✓	NoToolForLionMatingProcessBFCL/irrelevance_35: mating process of a species asked; only a dominant/recessive genetic-traits lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 788 OUT438
✓	NoToolForMitochondriaCountBFCL/irrelevance_29: number/count of mitochondria asked; only a cell-part FUNCTION lookup is available (wrong query direction).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 792 OUT191
✓	NoToolForPandaEvolutionaryHistoryBFCL/irrelevance_42: evolutionary history asked (a narrative fact); only a biodiversity-index calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 788 OUT315
–	NoToolForWhaleGeneSequenceBFCL/irrelevance_44: the actual gene sequence of a species asked; only a gene-sequence GENERATOR (needs numeric mutation rate + duration) is available.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'gene_sequencer' (generate gene sequences for evolutionary changes) directly answers the query - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
Tool Calling - Irrelevance: Color, Geometry & Population 11 passed	100.0%	IN40 858 OUT1 809
✓	NoToolForColorInComputerVisionBFCL/irrelevance_131: color meaning in computer vision asked; only a color-in-personality-psychology analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 714 OUT269
✓	NoToolForDeclarationSigningDateBFCL/irrelevance_135: signing date asked; only date-arithmetic (add days) and age tools are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 712 OUT52
✓	NoToolForElectricityDiscovererBFCL/irrelevance_140: who discovered electricity asked; only an electric-current calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 708 OUT192
✓	NoToolForHexColorMixBFCL/irrelevance_146: mixing two hex colors asked; only a prophet-details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 724 OUT670
✓	NoToolForIndependenceSigningDateBFCL/irrelevance_138: signing date asked; only an age calculator (needs birthdate + current date) and a date-adder are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 712 OUT54
✓	NoToolForMostVisitedMarketBFCL/irrelevance_161: most visited market asked; only a museum visitation-statistics lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 714 OUT92
✓	NoToolForNbaFinalWinnerBFCL/irrelevance_133: NBA final winner asked; only a historical-battle details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 715 OUT48
✓	NoToolForPicassoPaintingYearBFCL/irrelevance_151: year of a Picasso painting asked; only a sculpture-dimensions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 713 OUT125
✓	NoToolForRelativityAuthorBFCL/irrelevance_142: who proposed special relativity asked; only a chemical-element properties lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 717 OUT57
✓	NoToolForSculptureCreatorBFCL/irrelevance_155: sculpture creator asked; only a max-seating-capacity calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 713 OUT68
✓	NoToolForWhichColorsToMixBFCL/irrelevance_150: which colors to mix for a shade asked; only an RGB-to-Pantone converter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 716 OUT182
Tool Calling - Irrelevance: Cooking, Currency & Markets 6 passed2 failed	75.0%	IN58 360 OUT2 060
×	NoToolForBakerySourdoughBFCL/irrelevance_223: a BAKERY selling sourdough asked; only a grocery-shop product finder available -> no tool fits.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 3	Failed	IN13 499 OUT371
✓	NoToolForChickenRecipeBFCL/irrelevance_211: the RECIPE to cook chicken asked; only a cook-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 243 OUT206
✓	NoToolForChocolateCakeIngredientsBFCL/irrelevance_220: cake INGREDIENTS asked; only a cooking-TEMPERATURE tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 246 OUT205
✓	NoToolForCurrencyToGbpBFCL/irrelevance_233: CURRENCY conversion asked; only a WEATHER tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 248 OUT110
✓	NoToolForHowToBoilAnEggBFCL/irrelevance_212: best WAY to boil an egg asked; only a boiling-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 245 OUT349
✓	NoToolForLasagnaRecipeBFCL/irrelevance_219: the RECIPE for lasagna asked; only a recipe-CALORIE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 243 OUT49
✓	NoToolForLegExercisesBFCL/irrelevance_221: leg EXERCISES asked; only a grocery-list generator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 243 OUT169
×	NoToolForPetStoreBFCL/irrelevance_224: a PET store asked; only a GROCERY-store locator available -> no tool fits.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 6	Failed	IN25 393 OUT601
Tool Calling - Irrelevance: Dates & Elements 9 passed1 failed	90.0%	IN44 185 OUT1 700
✓	NoToolForHydrogenPropertiesBFCL/irrelevance_141: chemical properties of Hydrogen asked; only a scientist-contributions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 628 OUT323
✓	NoToolForLargestPlanetBFCL/irrelevance_139: largest planet asked; only a star-info lookup (needs a star name) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 628 OUT177
✓	NoToolForMostImportantProphetBFCL/irrelevance_147: most important prophet asked; only a two-color mixer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 628 OUT68
×	NoToolForMuseumVisitFrequencyBFCL/irrelevance_160: museum member visit frequency asked; only a single most-frequent-visitor lookup is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2	Failed	IN11 528 OUT493
✓	NoToolForPantoneOfSkyBlueBFCL/irrelevance_149: Pantone code of sky blue asked; only a paint-proportions calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 630 OUT177
✓	NoToolForPersonalityTypeBFCL/irrelevance_130: user asks what personality type they are; only a big-five calculator (which needs the user's rated answers) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 626 OUT83
✓	NoToolForStadiumSeatingCapacityBFCL/irrelevance_154: stadium seating capacity asked; only a sculpture-creator lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 629 OUT102
✓	NoToolForVicePresidentBFCL/irrelevance_136: Vice President asked; only a US-president-by-year lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 628 OUT49
✓	NoToolForWaterlooCasualtiesBFCL/irrelevance_132: casualty count of a battle asked; only a historical-event date lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 630 OUT189
✓	NoToolForWorldCupWinnerBFCL/irrelevance_134: sports World Cup winner asked; only a historical-battle outcome predictor is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 630 OUT39
Tool Calling - Irrelevance: Electromagnetism, Thermo & Data 9 passed1 failed	90.0%	IN41 504 OUT2 209
✓	NoToolForElectronRestMassBFCL/irrelevance_23: mass of an electron asked (a knowledge fact); only magnetic-field / current tools are available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 399 OUT30
✓	NoToolForFlightDurationBFCL/irrelevance_20: flight duration asked; only a currency converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 402 OUT123
✓	NoToolForFreezingPointBFCL/irrelevance_25: freezing point asked; only boiling-point calculators are available (wrong phase transition).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 409 OUT346
×	NoToolForGasAmountGeneratedBFCL/irrelevance_26: amount of gas generated asked; only a gas-PRESSURE calculator is available (wrong quantity).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2	Failed	IN10 861 OUT719
✓	NoToolForGiniCoefficientBFCL/irrelevance_48: Gini coefficient asked; only a GDP-fetch tool is available (different economic indicator).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 402 OUT169
✓	NoToolForHeatingWaterBFCL/irrelevance_27: heat energy to warm water asked. NOTE: the original tool (calculate_heat) is a fair match for this query (only the well-known specific heat is missing), which would make "no call" ambiguous; per human-baseline fairness it is deliberately NOT shown here (kept as a distractor in A3), so no visible tool fits and "no call" is unambiguously correct.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 413 OUT331
✓	NoToolForHexagonSidesBFCL/irrelevance_28: number of sides of a hexagon asked (basic geometry knowledge); only a boiling-point tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 400 OUT8
✓	NoToolForSymbolicMagneticFieldBFCL/irrelevance_22: magnetic field asked with SYMBOLIC values (distance 'r', current 'I'); the two magnetic-field tools require concrete numeric distance + current, so there is nothing to pass -> no fit.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 413 OUT210
✓	NoToolForTimeDifferenceBFCL/irrelevance_49: time-zone difference asked; only a geo-coordinate lookup tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 402 OUT223
✓	NoToolForWorldSeriesWinnerBFCL/irrelevance_53: sports trivia asked; only a SQL database-query runner is available (no database/query provided).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 403 OUT50
Tool Calling - Irrelevance: Entertainment, Sports & Knowledge 9 passed	100.0%	IN31 428 OUT1 055
✓	NoToolForBasketballResultBFCL/irrelevance_84: basketball game result asked; only a stock-price retrieval tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 492 OUT35
✓	NoToolForBestSoccerPlayerBFCL/irrelevance_79: best current soccer player asked; only a stock-market prediction tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 490 OUT150
✓	NoToolForCaliforniaGovernorBFCL/irrelevance_87: current Governor of California asked; only a court-case filing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 488 OUT41
✓	NoToolForCrimeThrillerMovieListBFCL/irrelevance_88: best crime-thriller movies of 2020 asked; only a document forgery detector is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 497 OUT306
✓	NoToolForFifaWorldCupWinnerBFCL/irrelevance_80: FIFA World Cup 2010 winner asked; only a stock-ticker lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 493 OUT13
✓	NoToolForFrenchTranslationBFCL/irrelevance_61: French translation of a word asked; only an SVM-model tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 492 OUT12
✓	NoToolForHorrorMovieListBFCL/irrelevance_77: horror movie recommendations asked; only an investment future-value tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 491 OUT377
✓	NoToolForPresidentialElectionWinnerBFCL/irrelevance_85: 2020 presidential election winner asked; only a criminal-case detail lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 493 OUT27
✓	NoToolForSportsNewsBFCL/irrelevance_76: latest sports news asked; only a portfolio investment-trend analyzer is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 492 OUT94
Tool Calling - Irrelevance: Finance & Economics 8 passed	100.0%	IN26 842 OUT1 543
✓	NoToolForAverageHousePriceBFCL/irrelevance_69: market-wide average house price asked; only a regression predictor (needs a trained model + feature vector, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 348 OUT68
✓	NoToolForCompoundInterestBFCL/irrelevance_70: compound-interest computation asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 374 OUT334
✓	NoToolForGoldPriceBFCL/irrelevance_78: today's gold spot price asked; only a bond-price calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 350 OUT46
✓	NoToolForHorrorMovieCatalogBFCL/irrelevance_81: horror movie recommendations asked; only a historical stock-price fetcher is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 351 OUT359
✓	NoToolForLossProjectionBFCL/irrelevance_73: a loss projection asked; only a revenue-prediction tool is available (revenue != loss).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 353 OUT488
✓	NoToolForPoundToKilogramBFCL/irrelevance_75: a mass unit conversion asked; only a portfolio-returns calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 349 OUT18
✓	NoToolForProfitMarginBFCL/irrelevance_71: profit margin asked; only an ROI calculator is available (ROI != margin) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 371 OUT138
✓	NoToolForStockNewsBFCL/irrelevance_82: stock news asked; only a capital-gains calculator is available (gains != news) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 346 OUT92
Tool Calling - Irrelevance: Finance Near-Matches 7 passed2 failed	77.8%	IN53 049 OUT7 862
✓	NoToolForBitcoinPriceBFCL/irrelevance_56: current bitcoin price asked; only a generic database record-select tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 428 OUT40
✓	NoToolForBookstoreLookupBFCL/irrelevance_55: which online bookstore sells a title asked; only an add-to-cart tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 433 OUT65
×	NoToolForBusinessRateOfReturnBFCL/irrelevance_74: rate of return asked; only a net-profit calculator is available (profit != rate of return).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2	Failed	IN15 169 OUT2 136
✓	NoToolForCoinFlipProbabilityBFCL/irrelevance_66: probability of a coin landing heads asked; only a standard-deviation tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 433 OUT103
✓	NoToolForCurrentInterestRateBFCL/irrelevance_83: the current market interest rate asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 427 OUT65
×	NoToolForExternalRateOfReturnBFCL/irrelevance_72: EXTERNAL rate of return asked; only an internal-rate-of-return (IRR) tool is available (ERR != IRR, even though cash flows are given) -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2	Failed	IN13 847 OUT3 962
✓	NoToolForFinalPriceWithoutBasePriceBFCL/irrelevance_60: final price after discount+tax asked, but the original price is never given, so the (domain-matching) calculateFinalPrice cannot be correctly called -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 447 OUT183
✓	NoToolForFrontendDevelopmentBFCL/irrelevance_62: how to build an e-commerce frontend asked; only a recommender-model builder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 433 OUT1 141
✓	NoToolForHighestGrossingMovieBFCL/irrelevance_54: highest-grossing movie of all time asked; only a movie search (requires a specific title + year, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 432 OUT167
Tool Calling - Irrelevance: Food, Time & Distance 3 passed1 failed1 skipped	75.0%	IN16 440 OUT872
✓	NoToolForCakeBakingMachineBFCL/irrelevance_218: a baking MACHINE/appliance asked; only a recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 708 OUT306
✓	NoToolForDrivingTravelTimeBFCL/irrelevance_215: driving TRAVEL TIME asked; only a restaurant-bill tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 712 OUT129
×	NoToolForLondonTimezoneNameBFCL/irrelevance_226: the NAME of London's timezone asked; only a local-TIME-by-timezone tool available (it needs the timezone as input and returns a time, not the timezone name) -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2	Failed	IN8 304 OUT296
–	NoToolForPizzaInBostonBFCL/irrelevance_213: a place for pizza asked; a cuisine+location restaurant_finder is available. REVIEW FLAG (fairness): restaurant_finder(cuisine=pizza/Italian, location=Boston) plausibly answers this, so a competent human might legitimately call it. BFCL labels it irrelevant; kept as such and marked Hard because the intended "no call" is debatable. No OTHER query here is a restaurant search, so the tool cannot wrongly satisfy the file's other cases.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'restaurant_finder' (cuisine+location) answers 'good place for pizza in Boston' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForTriangleHypotenuseBFCL/irrelevance_238: the HYPOTENUSE of a triangle asked; only a geo lat/long distance calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 716 OUT141
Tool Calling - Irrelevance: Geo, Travel & Sentiment 7 passed1 failed1 skipped	87.5%	IN157 657 OUT3 595
✓	NoToolForAverageTemperatureBmiBFCL/irrelevance_110: an average December temperature asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 398 OUT30
✓	NoToolForCaseLawyerBFCL/irrelevance_95: the lawyer of a case asked; only a case-outcome retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 401 OUT59
✓	NoToolForCurrentTimeSunriseBFCL/irrelevance_102: the current time asked; only a sunrise/sunset time calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 397 OUT32
✓	NoToolForFamousQuoteBFCL/irrelevance_119: a famous quote asked; only a sentiment-analysis tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 395 OUT32
✓	NoToolForFastestRouteBFCL/irrelevance_114: the fastest route asked; only a closest-airport finder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 399 OUT122
–	NoToolForRestaurantsInLondonBFCL/irrelevance_113: restaurants in London asked; only a generic points-of-interest locator is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'find_pois' (locate points of interest by category) answers 'restaurants in London' (restaurants are POIs) - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForSportsBrainImpactBFCL/irrelevance_120: the neurological impact of sports asked; only a caffeine neurological-impact tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 398 OUT248
✓	NoToolForTrafficConditionBFCL/irrelevance_112: the current traffic condition asked; only an address geocoder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 397 OUT49
×	NoToolForTravelTimeByCarBFCL/irrelevance_115: travel time by car asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 23 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 23	Failed	IN133 872 OUT3 023
Tool Calling - Irrelevance: Legal & Case Lookup 9 passed1 failed	90.0%	IN42 912 OUT1 367
✓	NoToolForAppleCaloriesBFCL/irrelevance_100: calories in an apple asked; only a litigation-cost calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 550 OUT39
✓	NoToolForBrainWeightBFCL/irrelevance_122: average human brain weight asked; only a per-patient neural-activity tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 552 OUT67
✓	NoToolForCaseJudgementBFCL/irrelevance_93: a case judgement asked; only a law-firm impactful-cases lister is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 550 OUT80
✓	NoToolForChineseRestaurantBFCL/irrelevance_97: a Chinese-restaurant recommendation asked; only a lawsuit-filing tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 553 OUT188
✓	NoToolForMovieRecommendationBFCL/irrelevance_116: a movie recommendation asked; only a word-count tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 551 OUT156
×	NoToolForNbaChampionBFCL/irrelevance_118: the 1996 NBA champion asked; only an NBA playoff-brackets display tool is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2	Failed	IN10 938 OUT349
✓	NoToolForPaintingEiffelSettlementBFCL/irrelevance_98: how long to paint the Eiffel Tower asked; only a lawsuit-settlement estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 554 OUT106
✓	NoToolForSecurityGuardSearchBFCL/irrelevance_92: finding a security guard asked; only a lawyer-search tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 560 OUT42
✓	NoToolForSphereVolumeBFCL/irrelevance_104: sphere volume asked; only a circular-park area tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 554 OUT216
✓	NoToolForTrafficLawsBFCL/irrelevance_99: traffic laws asked; only a lawsuit-search tool is available (lawsuits, not laws) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 550 OUT124
Tool Calling - Irrelevance: Math & Science Distractors 8 passed	100.0%	IN25 252 OUT1 352
✓	NoToolForDefiniteIntegralBFCL/irrelevance_9: definite integral asked; only an income-tax calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 159 OUT159
✓	NoToolForIntegralOfCubeBFCL/irrelevance_8: definite integral asked; only a string-to-int converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 152 OUT254
✓	NoToolForLatestMoviesBFCL/irrelevance_15: latest movie releases asked; only a physics final-velocity tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 146 OUT68
✓	NoToolForPerpendicularSlopeBFCL/irrelevance_3: slope of a perpendicular line asked; only a critical-points finder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 164 OUT189
✓	NoToolForPrimeFactorsBFCL/irrelevance_13: prime factorization asked; only a compound-interest calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 149 OUT95
✓	NoToolForRectangleAreaBFCL/irrelevance_6: rectangle area asked; only a circle-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 160 OUT125
✓	NoToolForTriangleAreaBFCL/irrelevance_0: triangle area asked; only a BMI tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 160 OUT169
✓	NoToolForUpwardAccelerationBFCL/irrelevance_14: acceleration asked (it is just gravity); the tempting max-height and final-velocity tools compute neither the acceleration nor what is asked -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 162 OUT293
Tool Calling - Irrelevance: Math & Statistics 5 passed1 failed	83.3%	IN21 941 OUT3 620
✓	NoToolForBinomialMeanBFCL/irrelevance_67: the MEAN of a binomial experiment asked; only p-value tools are available (they return a p-value, not the mean) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 028 OUT294
✓	NoToolForCivilLawComponentsBFCL/irrelevance_90: the components/categories of Civil Law asked; only a noise-complaint filing tool is available (filing a complaint does not explain law) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 000 OUT232
✓	NoToolForCoinTossMaxHeadsBFCL/irrelevance_63: how many heads CAN be obtained from tossing 3 coins (a count/maximum, not a probability) asked; only a probability-of-an-event calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 004 OUT106
×	NoToolForNullHypothesisStdDevBFCL/irrelevance_68: the STANDARD DEVIATION of a null-hypothesis test asked; only p-value tools are available (they return a p-value, not a standard deviation) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 897 OUT2 583
✓	NoToolForPowerOfTwoBFCL/irrelevance_59: raising a number to a power asked; only a linear-equation solver is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 003 OUT38
✓	NoToolForQuadraticEquationBFCL/irrelevance_57: solving a quadratic equation asked; only a genetic-algorithm optimizer and a LINEAR equation solver are available (a quadratic is not linear) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 009 OUT367
Tool Calling - Irrelevance: Medical, Social & Personality 6 passed3 skipped	100.0%	IN20 464 OUT936
✓	NoToolForBigMacCaloriesBFCL/irrelevance_123: calories of a Big Mac asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 409 OUT43
✓	NoToolForEconomicHappinessEffectBFCL/irrelevance_126: the effect of economic status on happiness asked; only a happiness-index lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 411 OUT113
✓	NoToolForFriendMbtiBFCL/irrelevance_129: a friend's MBTI asked; only a personality-type evaluator (needs trait percentages) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 409 OUT85
–	NoToolForLatestTechTrendBFCL/irrelevance_124: the latest technology trend asked; only a trending-topics-by-category tool is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_social_trends' (trending topics by category) answers 'latest trend in technology' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForMajorAirportsBFCL/irrelevance_117: major US airports asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 412 OUT224
–	NoToolForMotorNeuronDiseaseBFCL/irrelevance_121: motor neuron disease info asked; only a generic disease-info retriever is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'medical_records_get_disease_info' directly answers 'information on motor neuron diseases' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForPopularBooksBFCL/irrelevance_125: popular books by an author asked; only a recent-tweets retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 411 OUT259
–	NoToolForTwitterMoodBFCL/irrelevance_127: overall twitter mood asked; only a twitter sentiment-analysis tool is available -> BFCL ground truth is no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'sentiment_analysis_twitter' directly answers 'general mood of twitter regarding the new iPhone' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForVegetableServingsBFCL/irrelevance_128: daily vegetable servings asked; only a personality-test scorer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 412 OUT212
Tool Calling - Irrelevance: Mixed Distractors 8 passed	100.0%	IN26 008 OUT962
✓	NoToolForAirQualityBFCL/irrelevance_40: current air quality asked; only a plant-biomass calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 252 OUT125
✓	NoToolForFavoriteRestaurantBFCL/irrelevance_31: a favorite restaurant recommendation asked; only a cell-division simulator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 248 OUT45
✓	NoToolForGlobalWarmingDocumentaryBFCL/irrelevance_38: a documentary (video) asked; only a scientific-research-paper retriever is available (wrong medium).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 249 OUT49
✓	NoToolForLasagnaIngredientsBFCL/irrelevance_47: recipe ingredients asked; only a flight-timetable tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 249 OUT165
✓	NoToolForLionDominantTraitBFCL/irrelevance_34: dominant genetic trait of a species asked; only a gene-variant-FREQUENCY lookup is available (different genetic output).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 251 OUT117
✓	NoToolForMultinucleatedCellNameBFCL/irrelevance_30: name of a cell type with multiple nuclei asked; only a blood-cell classifier (by shape/size) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 255 OUT175
✓	NoToolForPicnicSpotBFCL/irrelevance_37: a picnic spot asked; only a local-fauna information tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 248 OUT221
✓	NoToolForTranslationBFCL/irrelevance_52: natural-language translation asked; only a file-compression tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 256 OUT65
Tool Calling - Irrelevance: Motion, Electricity & Misc 10 passed	100.0%	IN35 428 OUT2 270
✓	NoToolForCarProjectileRangeBFCL/irrelevance_16: projectile-style "how far will a car travel" asked with symbolic v/t/theta; the range tool needs concrete numeric velocity + angle -> nothing to pass -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 553 OUT435
✓	NoToolForConeVolumeBFCL/irrelevance_19: cone volume asked; only an investment-return calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 547 OUT383
✓	NoToolForCurrentClockTimeBFCL/irrelevance_17: current clock time asked; only a travel-time (distance/speed) calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 536 OUT82
✓	NoToolForElectronMassBFCL/irrelevance_24: mass of an electron asked (a knowledge fact); only an electric-current tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 537 OUT25
✓	NoToolForFibonacciOf20BFCL/irrelevance_51: fibonacci number asked; only a cryptocurrency-price tool is available (the fibonacci calculator lives in a different file so it cannot satisfy this query).1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 538 OUT151
✓	NoToolForForceAngleFromMomentumBFCL/irrelevance_18: angle of a force from momentum asked; only a vector-angle-from-XY-components tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 543 OUT290
✓	NoToolForHorrorMovieListBFCL/irrelevance_50: movie recommendations asked; only a Celsius-to-Fahrenheit converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 543 OUT329
✓	NoToolForMagneticFieldNearWireBFCL/irrelevance_21: magnetic field near a current-carrying wire asked; only an EM-wave-amplitude tool is available (magnetic-field calculators are deliberately kept in a different file).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 552 OUT313
✓	NoToolForSineOf45DegreesBFCL/irrelevance_45: sine of an angle asked; only a polygon-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 538 OUT145
✓	NoToolForTeslaPriceBFCL/irrelevance_46: product price of a car asked; only a currency-exchange-rate tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 541 OUT117
Tool Calling - Irrelevance: Music Theory & Lookup 8 passed	100.0%	IN25 588 OUT963
✓	NoToolForAncientGreekLawmakersBFCL/irrelevance_171: who established laws in Ancient Greece asked (history); only a musical-piece composer search is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 196 OUT160
✓	NoToolForBasketballGameAtVenueBFCL/irrelevance_169: who is playing a basketball game at a venue asked; only concert-locating tools exist.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 199 OUT72
✓	NoToolForBluesScalePatternBFCL/irrelevance_176: the interval pattern of a blues scale asked; only composer lookups are available (get_blues_scale lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 201 OUT242
✓	NoToolForDonQuixoteAuthorBFCL/irrelevance_172: author of the novel Don Quixote asked; only music-composition lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 194 OUT16
✓	NoToolForFamousUsComposersBFCL/irrelevance_170: most famous US composers asked (general knowledge, no specific piece); only a chord-progression creator + piece-specific composer lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 197 OUT195
✓	NoToolForGrammyBestAlbumBFCL/irrelevance_177: Grammy Award winner asked; only a song chord-progression lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 202 OUT83
✓	NoToolForPrimaryTriadsOfKeyBFCL/irrelevance_173: primary triads of a key asked; only a genre-common-chords finder is available (primary_triads lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 200 OUT105
✓	NoToolForUniversityClassAvailabilityBFCL/irrelevance_168: availability of a university class asked; only concert-availability / concert-search tools are available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 199 OUT90
Tool Calling - Irrelevance: Probability & Legal 4 passed1 failed	80.0%	IN17 050 OUT2 748
✓	NoToolForFaceCardProbabilityBFCL/irrelevance_64: probability of drawing a face card from a deck asked; only a COIN-toss heads probability tool is available (coins, not cards) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 798 OUT172
×	NoToolForHomeElectricityNeedBFCL/irrelevance_58: home electricity demand (kWh) asked; only a solar-panel COUNT calculator is available (it sizes panels, not electricity need, and needs sunlight hours not given) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN5 844 OUT693
✓	NoToolForNoiseComplaintProcessBFCL/irrelevance_91: how/whether to report a noise complaint asked; only a law-category listing tool is available (it lists legal categories, it does not file or advise on complaints) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 798 OUT67
✓	NoToolForRedMarbleCountBFCL/irrelevance_65: number of red marbles asked; only a determine-population tool is available (it returns a total population from probability + sample size, not a sub-category count) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN2 813 OUT1 485
✓	NoToolForTenantRightsBFCL/irrelevance_89: explaining tenant rights asked; only a contract-generation tool is available (generating a contract does not explain rights) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN2 797 OUT331
Tool Calling - Irrelevance: Recipes & Cooking 8 passed1 skipped	100.0%	IN27 658 OUT1 412
✓	NoToolForBestGameCharacterBFCL/irrelevance_206: best game character vs a dragon asked; only a best-WEAPON finder available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 461 OUT159
✓	NoToolForBestMovie2020BFCL/irrelevance_235: best MOVIE of 2020 asked; only a currency-exchange calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 458 OUT65
✓	NoToolForBuildingAPcBFCL/irrelevance_209: how to build a PC asked; only a cooking-recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 453 OUT797
✓	NoToolForFamousPlaceInParisBFCL/irrelevance_210: most famous PLACE in Paris asked; only a restaurant-by-dish search available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 455 OUT68
✓	NoToolForGameShortestPathBFCL/irrelevance_208: shortest in-game path asked; only a taxi-fare calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 463 OUT64
✓	NoToolForInchToMeterBFCL/irrelevance_234: LENGTH conversion (inches to metres) asked; only a CURRENCY converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 454 OUT66
✓	NoToolForLowestGameScoreBFCL/irrelevance_207: LOWEST game score asked; only a HIGHEST-score retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 458 OUT95
✓	NoToolForNewYorkPopulationBFCL/irrelevance_225: current POPULATION of New York asked; only a time-zone converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 456 OUT98
–	NoToolForQuickestFlightLondonTokyoBFCL/irrelevance_236: quickest way to fly London->Tokyo asked; a flight-duration tool is available. REVIEW FLAG (fairness): get_flight_duration plausibly answers this ("quickest ... by plane" ~= quickest flight duration), so a competent human might legitimately call it. Kept as BFCL labels it (irrelevant), but this item is borderline. Marked Hard because the intended "no call" is debatable.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_flight_duration' (with flight_type non-stop) plausibly answers 'quickest way ... by plane' - BFCL-mislabeled/borderline-relevant. Kept for traceability.	Skipped	IN0 OUT0
Tool Calling - Irrelevance: Restaurants & Travel 7 passed1 failed	87.5%	IN50 172 OUT1 040
✓	NoToolForBeijingSunsetBFCL/irrelevance_227: SUNSET time asked; only a SUNRISE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 315 OUT43
✓	NoToolForClosestItalianRestaurantBFCL/irrelevance_216: FIND the closest Italian restaurant asked; only a currency converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 314 OUT53
✓	NoToolForFindSushiRestaurantBFCL/irrelevance_214: FIND a sushi restaurant asked; only a tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 316 OUT54
✓	NoToolForNearestPharmacyBFCL/irrelevance_237: FIND the nearest pharmacy asked; only a route-to-a-known-location planner available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 316 OUT59
✓	NoToolForPetFriendlyTrainStationBFCL/irrelevance_230: a pet-friendly TRAIN STATION asked; only hotel search tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 315 OUT54
✓	NoToolForSushiRestaurantsTokyoBFCL/irrelevance_229: FIND sushi RESTAURANTS asked; only hotel-booking tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 316 OUT189
✓	NoToolForThaiRestaurantChicagoBFCL/irrelevance_231: FIND a Thai RESTAURANT asked; only a hotel-availability checker available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 317 OUT52
×	NoToolForYogaRetreatBFCL/irrelevance_232: a yoga RETREAT asked; only a HOTEL search tool available. REVIEW FLAG (fairness): a retreat is not a hotel, but hotel_search_find_hotels accepts arbitrary amenities (spa, vegetarian) and a location, so it is somewhat tempting. BFCL labels this irrelevant (retreat != hotel); kept as such and marked Hard because the boundary is borderline.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 6	Failed	IN26 963 OUT536
Tool Calling - Irrelevance: Sports & Games (set 1) 9 passed1 failed	90.0%	IN43 035 OUT1 632
×	NoToolForAssistAverageBFCL/irrelevance_185: assist average asked; only an average-SCORING tool is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2	Failed	IN10 899 OUT189
✓	NoToolForBestChessMoveBFCL/irrelevance_194: best next move asked; only a board-game-STATISTICS tool is available (the move-suggesting board_analyser lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 574 OUT88
✓	NoToolForChessArrangementCountBFCL/irrelevance_196: total theoretical arrangements asked; only a final-score calculator is available (possible_moves lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 574 OUT160
✓	NoToolForFootballTeamRankingBFCL/irrelevance_186: ranking of a team asked; only a season-MVP tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 570 OUT79
✓	NoToolForIceCreamStoreBFCL/irrelevance_201: find a physical store asked; only a video-game guide is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 567 OUT77
✓	NoToolForMostHeartsInPokerBFCL/irrelevance_199: who holds most of a suit asked; only a poker-dealing tool is available (get_highest_card_holder lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 574 OUT174
✓	NoToolForPlayerBiographyBFCL/irrelevance_189: a biographical "who is X" asked; only a top-ranked-athlete lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 567 OUT84
✓	NoToolForTeamRosterBFCL/irrelevance_192: full roster asked; only a top-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 569 OUT83
✓	NoToolForUnoRulesBFCL/irrelevance_198: the rules of a game asked; only a deck-shuffling tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 571 OUT627
✓	NoToolForWorldCupWinnerBFCL/irrelevance_191: who won a tournament asked (no team named); only a per-team match-stats tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 570 OUT71
Tool Calling - Irrelevance: Sports & Games (set 2) 8 passed1 failed	88.9%	IN38 324 OUT927
✓	NoToolForBlackjackAceRuleBFCL/irrelevance_200: a rules question asked; only a poker highest-card-holder tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 469 OUT123
✓	NoToolForBookAuthorBFCL/irrelevance_205: author of a book asked; only a video-game score calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 473 OUT19
✓	NoToolForChessTournamentWinnerBFCL/irrelevance_195: who won a tournament asked; only a board-position move-suggester is available (the game-history fetch_game_stats lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 470 OUT79
✓	NoToolForCreateJigsawBFCL/irrelevance_204: CREATE a puzzle asked; only a puzzle-SOLVING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 463 OUT124
×	NoToolForHaloPlayerRankBFCL/irrelevance_203: a player's RANK asked; only a player-SCORE lookup is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2	Failed	IN10 581 OUT195
✓	NoToolForLakersTopScorerBFCL/irrelevance_193: top scorer of a team asked; only a team-details tool (roster/results/upcoming, not scorers) is available (sports_team.get_top_scorer lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 468 OUT162
✓	NoToolForMonopolyWinnerBFCL/irrelevance_197: who won a casual game asked; only a possible-moves calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 468 OUT79
✓	NoToolForTopTennisPlayerBFCL/irrelevance_190: top ranked PLAYER asked; only a TEAM standing/ranking tool is available (the athlete-ranking get_top_ranked lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 467 OUT35
✓	NoToolForWorldSeriesWinnerBFCL/irrelevance_202: who won a real game asked; only a score calculator (from raw points) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 465 OUT111
Tool Calling - Irrelevance: Sports Statistics 8 passed1 failed	88.9%	IN42 684 OUT1 132
✓	NoToolForBasketballSeasonMvpBFCL/irrelevance_187: who won the MVP asked; only a team-ranking tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 511 OUT161
✓	NoToolForClippersRosterBFCL/irrelevance_179: roster of a team asked; only a two-team game-results tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 504 OUT60
✓	NoToolForCommonChordsInPopBFCL/irrelevance_174: most common chords in a pop song asked; only a key-signature primary-triads tool is available (find_common_chords lives in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 509 OUT198
×	NoToolForCricketMatchParticipantsBFCL/irrelevance_180: who is playing in cricket matches today asked; only a cricket-schedule tool (dates/matches, not participants) is available -> no tool fits.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 3	Failed	IN14 614 OUT295
✓	NoToolForLaLigaPlayersBFCL/irrelevance_181: who played in a whole league asked; only a two-named-team last-match-result tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 504 OUT182
✓	NoToolForMoonlightSonataComposerBFCL/irrelevance_175: composer of Moonlight Sonata asked; only a blues-scale generator is available here (all composer-lookup tools are isolated in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 507 OUT24
✓	NoToolForNbaMvpWinnerBFCL/irrelevance_184: who won an MVP award asked (no player named); only a per-player stats tool that requires a player name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 513 OUT103
✓	NoToolForTopAssistPlayerBFCL/irrelevance_178: top ASSIST player asked; only a top-GOAL-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 508 OUT49
✓	NoToolForWimbledonChampionBFCL/irrelevance_183: winner of a tournament asked (no person named); only a celebrity-info tool that requires a name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 514 OUT60
Tool Calling - Irrelevance: Time, Weather & Misc 3 passed1 skipped	100.0%	IN7 742 OUT338
✓	NoToolForCityDistanceKmBFCL/irrelevance_239: DISTANCE (km) between two cities asked; only a get_date tool that returns the TIME difference between two locations is available (its 'unit' arg mentions distance, but it computes time) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 584 OUT87
–	NoToolForCurrentTimeSydneyBFCL/irrelevance_228: current time in Sydney asked; a local-time-by-city tool is available. REVIEW FLAG (fairness): get_local_time(location='Sydney') directly answers this, so a competent human would legitimately call it. BFCL nonetheless files it under irrelevance; kept as such to preserve the source label, but this item is effectively RELEVANT and should be reviewed. Marked Hard because the "no call" expectation is contradicted by the tool itself.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_local_time' directly answers 'current time in Sydney' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForTomatoCaloriesBFCL/irrelevance_222: CALORIES in a tomato asked; only a grocery-item-details lookup available. REVIEW FLAG (fairness): item_details at 'detailed' level could arguably surface nutrition info, and it requires a store_location the user never gave; BFCL labels it irrelevant (it is a grocery-store item lookup, not a nutrition-facts service). Kept as such; marked Medium as borderline.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 580 OUT87
✓	NoToolForWriteABookBFCL/irrelevance_217: "can you write a book" asked; only a recipe-CREATION tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN2 578 OUT164
Tool Calling - Irrelevance: Weather, Environment & Services 9 passed1 failed	90.0%	IN75 987 OUT2 735
✓	NoToolForBestCo2ReductionBFCL/irrelevance_108: the best way to reduce CO2 asked; only a numeric emission-reduction estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 662 OUT245
✓	NoToolForBestMonthToVisitBFCL/irrelevance_101: best month to visit Hawaii asked; only an average-monthly-temperature tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 660 OUT224
✓	NoToolForCactusNutrientsBFCL/irrelevance_109: a plant's nutrient needs asked; only a weekly watering-needs calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 667 OUT497
✓	NoToolForCurrentTimeWeatherBFCL/irrelevance_103: the current time asked; only a current-weather forecast tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 661 OUT140
✓	NoToolForGardenSpeciesCompositionBFCL/irrelevance_107: garden species composition asked; only a soil sand/silt/clay composition analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 664 OUT300
×	NoToolForGmoYogaMatBFCL/irrelevance_111: a GMO yoga mat asked; only an eco-friendly product locator is available (GMO is not eco-friendly) -> no tool fits.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 9	Failed	IN43 013 OUT839
✓	NoToolForImpactfulCasesBFCL/irrelevance_94: a firm's impactful cases asked; only a single-case detail lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 671 OUT75
✓	NoToolForPaintingEiffelCaseResultBFCL/irrelevance_96: how long to paint the Eiffel Tower asked; only a law-case result lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 663 OUT109
✓	NoToolForPollinationFlowersBFCL/irrelevance_105: top pollination flower species asked; only a route elevation-profile plotter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 665 OUT176
✓	NoToolForTomatoFertilizerBFCL/irrelevance_106: best fertilizer for tomatoes asked; only a soil-type analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 661 OUT130
Tool Calling - Parallel Calls: Art, Music & Local Services 1 passed9 failed1 skipped	10.0%	IN213 221 OUT8 855
–	CalculateDrivingDistancesShortestRouteBFCL/parallel_38: two calc_distance calls — New York→Washington DC and Los Angeles→San Francisco. Tightened per methodology: the prompt explicitly enables "shortest route", so shortest_route == true is required (schema default is false).0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Skipped: Ambiguous scope (item flaw, not model behavior): the trailing 'with optional parameter shortest route enabled' can reasonably apply to only the second leg (its nearest clause) or to both; the BFCL ground truth requires it on both, but a competent human would not answer deterministically. Kept for traceability.	Skipped	IN0 OUT0
×	CheckInstrumentAvailabilityInTwoCitiesBFCL/parallel_42: two check_instrument_availability calls — Yamaha P125 in Berlin and Madrid.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN16 025 OUT580
×	FindMusicStoresInTwoCitiesBFCL/parallel_41: two find_music_instrument_store calls — San Francisco and Chicago; instruments {Yamaha acoustic guitar, Kawai piano} in any order and either casing.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN20 955 OUT858
×	FindRockAndJazzConcertsInTwoCitiesBFCL/parallel_43: four concert_finder calls — {San Francisco, New York} x {rock, jazz} over the next month. time_period may be omitted (defaults to 30 days ≈ "next month"). Isolated from concert_find_nearby (parallel_44) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN12 560 OUT654
×	GetExhibitionDetailsForTwoMuseumsBFCL/parallel_40: two museum_exhibition_detail calls — 'Wonder of Nature' at the Louvre and 'Age of Reptiles' at the British Museum. visitor_type {child, adult} (unordered). Tightened per methodology: the prompt explicitly asks for cost "for children and adult", so both visitor types are required (schema default is ['adult'] only).1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2	Failed	IN32 191 OUT1 262
×	GetMostPlayedPopAndRockSongsBFCL/parallel_45: two musicCharts_getMostPlayed calls — most played Pop and Rock song in Australia. Optional duration omitted (ground truth accepts the schema default of 0).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN11 567 OUT458
×	GetMuseumInfoForTwoCitiesBFCL/parallel_39: two museum_info_get_info calls — Washington D.C. and Paris; details {Opening hours, Adult tickets, Child tickets} in any order.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN20 903 OUT894
×	GetRgbAndHexForThreeColorsBFCL/parallel_37: three color_converter_get_color_info calls — navy, purple, maroon; each with {RGB, HEX} (unordered).1 AI query 21 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 21 expectedCount = 3	Failed	IN41 103 OUT1 790
×	GetTeamRankingsInTwoLeaguesBFCL/parallel_47: two get_team_ranking calls — Barcelona in UEFA Champions League and Manchester United in La Liga.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN11 573 OUT292
×	MixPaintColorsForTwoRequestsBFCL/parallel_36: two paint_color_mixture calls — watercolor magenta and acrylic navy.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 11 expectedCount = 2	Failed	IN38 574 OUT1 571
✓	WinningPercentageForTwoTeamsTwoSeasonsBFCL/parallel_46: four calculate_winning_percentage calls — {Lakers, Bulls} x {2018, 2020}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 770 OUT496
Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 1) 3 passed10 failed	23.1%	IN222 611 OUT11 211
✓	AverageFourNumberSetsBFCL/parallel_132: four calculate_average calls — one per list of numbers (element order preserved).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 768 OUT335
×	CalculateBmiFourPeopleMetricBFCL/parallel_134: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt states "All measurements are in the metric system", so unit == "metric" is required (also the default).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN14 033 OUT905
×	CalculateCityDistancesInKilometersBFCL/parallel_135: three geo_distance_calculate calls — NY->LA->Miami->NY, in kilometers. Tightened: the tool defaults to miles but the prompt asks "in kilometers", so units == "kilometers" is REQUIRED (omission would return miles, contradicting the request — the ground truth's "omit" option is unsafe here).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 3	Failed	IN13 099 OUT834
×	CalculateGpsDistancesAlongRouteBFCL/parallel_133: three calculate_distance calls over GPS coordinate pairs (Paris->Rome->Athens-> Giza), each in kilometers. Uses the coordinate-based tool (coord1/coord2), not the city-name distance tools that are also visible.1 AI query 13 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 13 expectedCount = 3	Failed	IN54 665 OUT1 462
×	CompareAverageTemperatureNyAndLaBFCL/parallel_128: two average_temperature calls — New York 7 days in Fahrenheit, Los Angeles 7 days in Celsius. Tightened: the prompt explicitly requests Fahrenheit for NY, so temp_unit == "Fahrenheit" is required (also the default); Celsius is non-default and required for LA.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 926 OUT550
×	CreateTwoHistogramsWithFiveBinsBFCL/parallel_129: two create_histogram calls — one per data set, 5 bins each (data order preserved).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 817 OUT758
×	EstimateElephantPopulationThreeScenariosBFCL/parallel_125: three elephant_population_estimate calls — current 500 at 2%, 1.5%, 2.5% over 10y.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN13 694 OUT945
×	FindItalianRestaurantsInNyAndLaBFCL/parallel_130: two find_restaurants calls — New York and Los Angeles, Italian, 4 results each, dietary_requirements [vegan, gluten-free]. Distinct from restaurant_find_nearby (this tool carries food_type + number, which the prompt supplies, so it is the correct choice here).1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2	Failed	IN29 150 OUT1 250
×	FindRestaurantsByDietInThreeCitiesBFCL/parallel_127: three restaurant_find_nearby calls — New York, Los Angeles, Chicago, each with dietary_preference [Vegan, Gluten-free, Dairy-free]. List order follows the prompt (matches ground truth).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN13 511 OUT854
✓	PredictEvolutionDarwinAndLamarckBFCL/parallel_126: two prediction_evolution calls — African Elephant over 5000y, Darwin then Lamarck. Tightened per methodology: the prompt explicitly says "using the Darwin model", so model == "Darwin" is required for the first call (ground truth also accepts omission via the 'Darwin' default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 385 OUT504
✓	ProjectPopulationGrowthJapanAndIndiaBFCL/parallel_124: four population_projections calls — Japan/10y and India/20y, each at the current (default) growth rate and at an explicit rate. "current growth rate" maps to the default (growth_rate omitted, accepts null or 0.01); the explicit 1.5%/2.1% rates are required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 710 OUT1 098
×	RouteThreeLegsAvoidingTollsBFCL/parallel_131: three map_routing_fastest_route calls — SF->Palo Alto->San Jose->home, all avoiding tolls. Tightened: the prompt explicitly says "avoiding toll roads", so avoid_tolls == true is required for every leg (non-default value stated in the prompt).1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN18 843 OUT881
×	ShortestCityDistanceBusWithAndWithoutTransferBFCL/parallel_136: two city_distance_find_shortest calls — NY->LA by bus, first without transfer then allowing transfer. Tightened: the prompt explicitly says "using a bus", so transportation == "bus" is required for both (also the default). allow_transfer: false/omitted for the first, true for the second.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 010 OUT835
Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 2) 14 failed	0.0%	IN296 548 OUT14 177
×	CalculateAreaOfThreeTrianglesBFCL/parallel_143: three calc_area_triangle calls — (10,5), (15,7), (20,10).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN23 797 OUT516
×	CalculateBinomialProbabilityBasketballBFCL/parallel_161: three calculate_binomial_probability calls — basketball shooter at 60%: (10,7), (15,10), (20,15). probability_of_success == 0.6 is required (prompt states 60%). ISOLATED here (away from prob_dist.binomial in ToolCalling-Parallel-D3): both tools compute binomial probability, and this prompt does not name a function, so having both visible would make the choice genuinely ambiguous. Kept apart so exactly one correct tool is available.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN14 210 OUT1 403
×	CalculateBmiInMetersForThreePeopleBFCL/parallel_138: three calculate_BMI calls (weight kg, height meters) — John, Sarah, Mike.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN19 667 OUT844
×	CalculateCelestialDistancesInMilesBFCL/parallel_149: two calculate_distance calls between celestial bodies (Mars-Venus, Mars-Jupiter), in miles. Uses the body-name distance tool (body1/body2). Tightened: prompt asks "in miles", so unit == "miles" is required (tool default is kilometers, so omission would be wrong).1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 2	Failed	IN35 751 OUT1 519
×	CalculateClockAnglesAtThreeTimesBFCL/parallel_145: three calculate_clock_angle calls — 3:15, 8:20, 11:50. Tightened: the prompt asks for results "rounded to 2 decimal places", so round_to == 2 is required (also the default).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN19 555 OUT1 315
×	CalculateFactorialOfFourNumbersBFCL/parallel_144: four math_factorial calls — 5, 3, 4, 2.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN14 180 OUT506
×	CalculateLightTravelTimeThreeLegsBFCL/parallel_147: three light_travel_time calls — 4.22, 6.1, 5.88 light years. Tightened: the prompt explicitly states the speed of light (299792458 m/s), so speed_of_light == 299792458 is required (also default).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN14 174 OUT1 064
×	CalculateSpeedForCarBicycleTrainBFCL/parallel_148: three calculate_speed calls — car 500m/25s in km/h, bicycle 1000m/200s in m/s, train 10000m/600s in km/h. Each output unit is explicitly named in the prompt, so to_unit is required.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN19 809 OUT912
×	FetchAllDataFieldsForTwoEmployeesBFCL/parallel_139: two employee_fetch_data calls — Tech Solutions, employees 12345 and 67890, each fetching all four data fields (order follows the prompt / ground truth).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN24 674 OUT857
×	FindDiCaprioDramaAndComedyMoviesBFCL/parallel_140: two imdb_find_movies_by_actor calls — Leonardo DiCaprio, Drama in 2010 and Comedy in 2012. Tightened: the prompt explicitly names each category, so category is required for both.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN18 681 OUT768
×	GetImaxAndTwoDTheaterReleasesBFCL/parallel_141: two get_theater_movie_releases calls — New York IMAX over 7 days, Los Angeles 2D over 14 days. Tightened: the prompt explicitly names each format, so format is required for both.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN18 695 OUT1 251
×	PlotTwoSineWavesBFCL/parallel_146: two plot_sine_wave calls with all five parameters supplied by the prompt.1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2	Failed	IN30 541 OUT953
×	SortFourListsAscendingAndDescendingBFCL/parallel_137: eight array_sort calls — four lists, each sorted ascending AND descending (list element order preserved as given).1 AI query 16 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 8	Failed	IN16 600 OUT1 479
×	UpdateTwoCustomersNameAndEmailBFCL/parallel_142: two update_user_info calls — users 12345 and 67890, both set name "John" and email "example@.com". update_info is a nested object. Tightened: the prompt names the 'CustomerInfo' database, so database == "CustomerInfo" is required (also the default).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN26 214 OUT790
Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 3) 1 passed10 failed	9.1%	IN165 976 OUT8 706
×	AreaUnderCurveForTwoPolynomialsBFCL/parallel_150: two mathematics_calculate_area_under_curve calls — polynomial [3,-2,1] over [-1,2] and [1,0,-1] over [0,3] (coefficient/limit order preserved).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 149 OUT724
×	BinomialProbabilitiesThreeGamesBFCL/parallel_160: three prob_dist_binomial calls — (20,5), (50,15), (100,30), each with p == 0.3 (prompt states a 0.3 win probability). The prompt explicitly names 'prob_dist.binomial'; the colliding calculate_binomial_probability tool is kept in a separate script (ToolCalling-Parallel-D2).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN12 635 OUT1 027
×	CalculateBmiSystemFourPeopleBFCL/parallel_154: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt says "use the metric system for all calculations", so system == "metric" is required (also the default).1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4	Failed	IN18 567 OUT1 142
✓	DiceRollProbabilitiesThreeScenariosBFCL/parallel_159: three probability_dice_roll calls — roll 4 three times and roll 2 twice on a six-sided die (die_sides default 6, accepts omission), then roll 7 twice on an eight-sided die (die_sides == 8 required, non-default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 962 OUT696
×	GenerateNormalVariatesTwoConfigsBFCL/parallel_158: four random_normalvariate calls — two at (mu 5, sigma 2) and two at (mu 10, sigma 3). Duplicate calls: each configuration must appear exactly twice (multiplicity 2).1 AI query 20 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 20 expectedCount = 4	Failed	IN30 788 OUT1 411
×	PredictPricesForFourHousesBFCL/parallel_157: four predict_house_price calls — New York (3/2/1500), Los Angeles (4/3/2000), Chicago (2/1/1200), Miami (3/2/1800).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN13 295 OUT963
×	RaiseTwoNumbersToPowersBFCL/parallel_152: two math_power calls — 3^5 and 2^3, no modulus (mod omitted / null).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN11 882 OUT356
×	RunLinearRegressionWithAndWithoutStandardizationBFCL/parallel_155: two run_linear_regression calls — same predictors [Age, Income, Education] and target "Spending Score", first without standardization then with. standardize is explicitly requested (false then true), so it is required for both.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 241 OUT375
×	TrainRandomForestFourConfigsBFCL/parallel_156: four random_forest_train calls on "data_random_forest" — (100,10), (200,20), (300,30), (400,40). The prompt explicitly names this function, disambiguating it from train_random_forest_classifier visible in this script.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN13 349 OUT957
×	TrainRfClassifierTwoConfigsBFCL/parallel_153: two train_random_forest_classifier calls — dataset1 (depth 10, 100 trees) and dataset2 (depth 20, 200 trees). The prompt explicitly names this function, disambiguating it from the other random-forest training tool (random_forest_train) visible in this script.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 215 OUT396
×	TriangleAreasInDifferentUnitsBFCL/parallel_151: three geometry_area_triangle calls in different units — (15,20) square meters, (25,30) square feet, (35,40) square inches. First unit is the default (accepts omission / square meters); feet and inches are non-default and required.1 AI query 7 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 3	Failed	IN20 893 OUT659
Tool Calling - Parallel Calls: Finance, Stats & Legal 3 passed10 failed	23.1%	IN234 528 OUT11 900
✓	BuildTwoDecisionTreeClassifiersBFCL/parallel_12: two model_DecisionTreeClassifier calls — gini/depth 5 and entropy/depth 10, both random_state 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 444 OUT245
×	CalculateFutureValueForThreeTermsBFCL/parallel_18: three financials_calculate_future_value calls — years 1, 5, 10 (same 1000 @ 0.05).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 3	Failed	IN26 543 OUT726
×	CalculatePresentValueForThreeTermsBFCL/parallel_14: three calculate_present_value calls — years 10, 20, 30 (same payment 1000, rate 0.05).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN18 955 OUT824
×	CapitalGainsTaxForTwoStatesBFCL/parallel_15: two calculate_capital_gains_tax calls — CA (15000/25000) and FL (20000/50000).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN18 161 OUT1 670
✓	ComputePythagorasForTwoTrianglesBFCL/parallel_10: two math_pythagoras calls — sides (3,4) and (5,12).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 387 OUT260
×	ConfidenceIntervalForTwoSampleSizesBFCL/parallel_13: two confidence_interval_calculate calls — sample sizes 50 and 150. Tightened per methodology: the prompt explicitly requests "95% confidence interval", so confidence_level == 0.95 is required (schema default is 0.9, but the prompt states the value).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN18 203 OUT1 042
✓	CourtCaseStatusAndTrialDatesBFCL/parallel_22: four court_info_get_case_status calls — cases XY1234 (LA) and GH5678 (Orange County), each queried for status (details may be omitted via the 'status' default) and for trial_date.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 758 OUT1 136
×	GetStockPriceAndVolumeForThreeCompaniesBFCL/parallel_17: three get_stock_data calls — AAPL, GOOG/GOOGL, MSFT; each with data_points {price, volume} (unordered).1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3	Failed	IN24 471 OUT925
×	LoanEligibilityForTwoInstitutionsBFCL/parallel_20: two loan_eligibility_check calls — HSBC (500k/100k) and Wells Fargo (700k/120k).1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 2	Failed	IN35 254 OUT1 138
×	MortgagePaymentForThreeLoanTermsBFCL/parallel_19: three calculate_mortgage_payment calls — loan terms 15, 20, 30 (400000 @ 0.04).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN13 601 OUT1 651
×	PredictHousePriceInTwoCitiesBFCL/parallel_11: two ml_predict_house_price calls — New York 3000 sq ft, Los Angeles 4000 sq ft.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN17 853 OUT1 082
×	ReturnOnInvestmentGainAndLossBFCL/parallel_16: two calculate_return_on_investment calls — gain 500 on 2000, loss -1000 on 5000.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN17 959 OUT535
×	SearchMoneyLaunderingConvictionsBFCL/parallel_21: two law_crimes_search calls — money laundering in San Francisco 2019 and Texas 2018.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN17 939 OUT666
Tool Calling - Parallel Calls: Legal, History & Environment 3 passed10 failed1 skipped	23.1%	IN259 771 OUT12 261
✓	BookTwoConnectingFlightsBFCL/parallel_30: two book_flight calls — San Francisco→Tokyo (2022-05-03) and Tokyo→Sydney (2022-05-18).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 061 OUT235
×	CalculateAlimonyForTwoDurationsBFCL/parallel_23: two alimony_calculator_ca_calculate calls — durations 10 and 20 years (10000/3000 income).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN14 102 OUT919
✓	CalculateWasteForFamilyAndBachelorBFCL/parallel_29: two waste_calculation_calculate calls — LA family (2 adults, 2 children) and New York bachelor (1 single). The population argument is a nested object.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 056 OUT538
×	FetchTwoHistoricalFactsBFCL/parallel_31: two history_fact_fetch calls — Treaty of Paris and Magna Carta. Optional depth/year omitted (ground truth accepts the schema defaults).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 859 OUT622
×	FindClassicalConcertsWithParkingBFCL/parallel_44: two concert_find_nearby calls — classical concerts near Berlin and Paris next Friday. Tightened per methodology: the prompt explicitly asks for concerts "with available parking", so amenities == ['Parking'] is required (this also matches the schema default, but the prompt states it). Isolated from concert_finder (parallel_43) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 16 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 2	Failed	IN50 734 OUT1 495
–	FindCourtCasesByNumberAndTypeBFCL/parallel_26: four court_case_find calls — the cross product of case numbers {67813, 71249} and types {Civil, Criminal}; case_number is a single-element array; Civil may be omitted via its default. FLAGGED as borderline (see report): the prompt "case numbers '67813','71249' for type 'Civil' and 'Criminal'" is ambiguous about the pairing; BFCL ground truth expects all four combinations, kept faithful.0 AI queries 0 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL Skipped: Ambiguous call decomposition (item flaw, not model behavior): 'case numbers 67813, 71249 for type Civil and Criminal' has no single deterministic reading - two calls (both numbers per type, case_number as a list) is as valid as the BFCL 2x2 = four single-number calls; a competent human would not answer deterministically. Kept for traceability.	Skipped	IN0 OUT0
×	FindNatureReservesNearTwoCitiesBFCL/parallel_27: two nature_reserve_find_nearby calls — Berkeley (10km, picnic tables + restrooms) and Tokyo (5km, playgrounds + biking trails). Amenity list order is unordered.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN24 893 OUT1 532
×	GetCaseDetailsWithHistoryAndLitigantsBFCL/parallel_24: two law_case_get_details calls — cases 28473 and 64725. Tightened per methodology: the prompt explicitly asks for "their history and details of litigants", so include_history == true and include_litigants == true are required (schema defaults are false).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN24 740 OUT747
×	GetEventsForTwoPresidenciesBFCL/parallel_32: two us_history_events_by_presidency calls — Abraham Lincoln and George Washington. Optional start_year/end_year omitted (ground truth accepts the schema defaults).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 821 OUT688
×	GetPresidentsAndVicePresidentsByYearBFCL/parallel_33: four get_president_and_vp calls — president for 1980 and 2016, vice president for 1975 and 2011.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN14 801 OUT938
×	GetReligionInfoForTwoEmpiresBFCL/parallel_35: two ancient_empires_get_religion_info calls — Persian Empire and Mauryan Empire. Tightened per methodology: the prompt explicitly asks for "religious history and influences", so include_influences == true is required (schema default is false).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 907 OUT1 254
×	GetTemperatureForTwoCitiesBFCL/parallel_28: two get_current_and_future_temperature calls — Seattle and Los Angeles, next 3 hours.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 835 OUT257
×	LookupLawsuitsForTwoYearsBFCL/parallel_25: two lawsuit_lookup calls — Dara Inc for years 2019 and 2018.1 AI query 14 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 14 expectedCount = 2	Failed	IN47 904 OUT1 397
✓	TrackChristianityInTwoRegionsBFCL/parallel_34: two religion_history_track calls — Christianity in Egypt and Turkey, 100–1500 A.D.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 058 OUT1 639
Tool Calling - Parallel Calls: Markets & Environment 4 passed9 failed	30.8%	IN209 891 OUT10 431
×	CalculateCagrForTwoInvestmentsBFCL/parallel_176: two calculate_cagr calls — (5000,7000,5) and (8000,12000,3).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN27 783 OUT1 051
✓	CalculateEmissionSavingsForSolarAndWindBFCL/parallel_186: two calculate_emission_savings calls — solar(12 months) and wind(8 months), Midwest region. Prompt states the Midwest region explicitly -> region required (accept "Midwest"/"Midwest region").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 562 OUT375
×	CompareFutureValueWithAndWithoutContributionsBFCL/parallel_182: two finance_calculate_future_value calls — (5000,0.07,10, no contribution) and (3000,0.06,10, contribution 200). First scenario states "no additional contributions" -> null or 0.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 523 OUT1 172
×	ComputeEuclideanNormForThreePointsBFCL/parallel_183: three math_hypot calls — (5,7), (10,15), (20,25). z omitted or 0 (2D points).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 3	Failed	IN13 168 OUT519
×	EstimatePopulationsForTwoSpeciesAcrossTwoYearsBFCL/parallel_185: four estimate_population calls — Bengal Tigers/India and African Elephants/Kenya for 2021 and for 2023. NOTE: upstream ground truth's third entry lists year "" (omitted -> default 2020), which is an upstream error: the prompt explicitly asks for 2023 for "both these species". We tighten both 2023 estimates to year == 2023 (fair deterministic answer).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN14 210 OUT1 251
×	FindRootsForTwoQuadraticEquationsBFCL/parallel_184: two algebra_quadratic_roots calls — (3,7,2) and (5,-4,1).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 406 OUT857
×	GetClosingPricesForTwoCompaniesOnTwoDatesBFCL/parallel_178: four get_stock_price (by-date variant) calls — the 2x2 cross product of {Microsoft, Apple} x {2022-01-01, 2022-02-01} on NASDAQ. Upstream ground truth's first entry loosely lists company_name as [Microsoft, Apple]; the deterministic human answer is the four distinct (company, date) combinations, which we require. exchange NASDAQ (default) may be omitted.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN14 231 OUT975
×	GetDetailedAirQualityForThreeDaysBFCL/parallel_187: three get_air_quality calls — New York City, detail true, for 2023-05-05, 05-04, 05-03. Prompt asks for PM2.5/PM10/ozone/sources -> detail == true; "today/yesterday/day before" with today = May 5, 2023.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN19 271 OUT760
✓	GetFourPriceTypesForTwoCompaniesBFCL/parallel_180: eight stock_price calls — {Microsoft, Apple} x {Open, Close, High, Low}, days 30. Prompt explicitly lists all four data types, so data_type must be set explicitly (required) to distinguish the eight calls; omission (default 'Close') cannot represent the four requested types.1 AI query 8 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL	Passed	IN9 442 OUT659
×	GetPricePerOunceForFourMetalsBFCL/parallel_177: four get_metal_price calls — gold, silver, platinum, palladium, measure 'ounce'. Prompt asks "price per ounce" -> measure == "ounce" required.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN13 970 OUT682
✓	GetStockPricesForFourCompaniesAndDurationsBFCL/parallel_181: four get_stock_prices calls — Apple(1 week), Microsoft(2 weeks), Amazon(3 weeks), Tesla(1 month). Each 'companies' argument is a single-element list.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 838 OUT257
✓	GetTrafficInfoForThreeLegsWithModesBFCL/parallel_188: three get_traffic_info calls — NY->LA (driving), LA->SF (bicycling), SF->NY (transit). First leg explicitly says "driving" -> mode == "driving" required (tightened; also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 763 OUT359
×	ProjectFutureValueForThreeStocksBFCL/parallel_175: three portfolio_future_value calls — AAPL(5000,0.07,5), MSFT(8000,0.06,7), AMZN(10000,0.08,10).1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 3	Failed	IN44 724 OUT1 514
Tool Calling - Parallel Calls: Mathematics & Physics (set 1) 5 passed8 failed	38.5%	IN162 627 OUT8 020
×	AreaOfTwoRightTrianglesInMetersBFCL/parallel_68: two calculate_area calls — right triangles base/height 12x15 and 18x24 in meters. unit required (tool default is 'cm'; the prompt states meters/square meters).1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 2	Failed	IN16 751 OUT578
✓	CheckPlayerStatusAcrossThreeSeasonsBFCL/parallel_49: three player_status_check calls — player 3142 in RocketLeague for seasons 2017/2018/2019. Seasons are explicitly given, so each is required (default 2023 not applicable).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 075 OUT448
✓	CircumferenceOfFourCirclesInCmBFCL/parallel_65: four calculate_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt gives every radius in cm (e.g. "5cm") and the tool default unit is 'm', so a correct call must pass unit == 'cm'/'centimeter' (ground truth requires it on the first call and merely allows omission on the rest; we require it consistently since it is stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 217 OUT563
×	ConvertEuroAndPoundsToDollarsBFCL/parallel_59: two get_conversion_cost calls — 15000 Euro->dollars and 200 pounds->dollars.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 245 OUT474
×	FinalVelocityForFourDropsBFCL/parallel_81: four calculate_final_velocity calls — heights 10/20/15/25, initial_velocity 0 (stated), gravity 9.81 (default, may be omitted).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN13 445 OUT1 055
×	FindItalianAndJapaneseRestaurantsBFCL/parallel_52: two restaurant_finder calls — Italian in New York (preferences ['Vegetarian']) and Japanese in Los Angeles (delivery; ['Delivery'] or omitted since default is ['Delivery']).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN16 857 OUT566
×	FindRootsOfTwoQuadraticsAlgebraBFCL/parallel_62: two algebra_quadratic_roots calls — 3x^2+4x+2 and 5x^2-7x+3.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 461 OUT826
×	FindSoupAndVeganSaladRecipesBFCL/parallel_51: two recipe_search_find calls — Chicken Noodle Soup (diet omitted/default 'Keto') and a Vegan Salad (diet == 'Vegan', dish name accepts several phrasings).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN16 697 OUT554
×	GcdOfJohnAndMaryNumbersBFCL/parallel_80: two math_gcd calls — gcd(36,48) and gcd(60,96). Same tool as parallel_76.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN21 825 OUT696
✓	GcdOfTwoPairsBFCL/parallel_76: two math_gcd calls — gcd(45,60) and gcd(81,27).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 882 OUT146
×	GetPokemonMovesAndCheckSolarBeamBFCL/parallel_48: two PokemonGO_get_moves calls — Pikachu's move list (move omitted/default 'Run') and a check whether Bulbasaur can learn 'Solar Beam'.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 158 OUT890
✓	PrimeFactorsFormattedAndArrayBFCL/parallel_74: four get_prime_factors calls — numbers 4567 and 7890, each formatted true and false (user wants "a formatted string as well as an array").1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 164 OUT1 087
✓	SaveGameProgressEasyAndHardBFCL/parallel_50: two game_save_progress calls — stage 7 easy and stage 3 hard; level omitted/default 'user'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 850 OUT137
Tool Calling - Parallel Calls: Mathematics & Physics (set 2) 3 passed10 failed	23.1%	IN225 047 OUT12 201
×	AreaOfThreeCirclesInMetersBFCL/parallel_66: three geometry_area_circle calls — radii 5/10/15 in meters (default is meters, so units may be omitted).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN18 243 OUT786
✓	BookTwoHotelRoomsBFCL/parallel_57: two hotel_booking_book calls — Sheraton (NY, 2 adults/1 child) and Marriott (LA, 1 adult/2 children).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 387 OUT352
×	CheckAvailabilityForTwoHotelsBFCL/parallel_56: two hotel_booking_check_availability calls — Queens Hotel (Berlin) and Royal Hotel (Paris), each with check-in/out dates (several date formats accepted).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN23 370 OUT898
×	CircumferenceOfFourCirclesInMetersBFCL/parallel_70: four geometry_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt asks for output in meters and gives radii in m, but the tool default unit is 'cm', so a correct call must pass units == 'm'/'meters' (ground truth requires it on the first call and only allows omission on the rest; we require it consistently since it is explicitly stated).1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 4	Failed	IN13 140 OUT653
×	DerivativeOfPolynomialAndItsDerivativeBFCL/parallel_71: two calculate_derivative calls — 3x^3-2x^2+5x-7 at x=4, then its derivative 9x^2-4x+5 at x=2.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN21 609 OUT1 017
×	FinalVelocityForThreeDragRaceRoundsBFCL/parallel_83: three final_velocity calls — (v0=0,a=5,t=10), (v0=10,a=7,t=8), (v0=20,a=4,t=12).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN18 689 OUT892
×	FindGroceryStoresByCriteriaBFCL/parallel_55: two grocery_store_find_by_criteria calls — NYC ['24 hours'] and San Diego ['Home Delivery'].1 AI query 14 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 14 expectedCount = 2	Failed	IN39 409 OUT1 672
×	GetTwoCookingRecipesBFCL/parallel_53: two get_cooking_recipe calls — Lasagne Bolognese for 4 and Caesar Salad for 2.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN22 435 OUT1 830
×	GetTwoExchangeRatesBFCL/parallel_58: two get_exchange_rate calls — USD->AUD and USD->CAD.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN17 269 OUT485
×	HighestCommonFactorOfFourPairsBFCL/parallel_77: four math_hcf calls — (45,60), (90,120), (36,48), (72,96). FLAGGED (see final report): the prompt is phrased as chaining ("use that result to find the HCF with...") but the ground truth treats it as four INDEPENDENT parallel calls on the literal pairs; a human might interpret it as sequential composition. Kept BFCL-faithful (four independent literal-pair calls) since parallel semantics preclude feeding one call's result into another.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 4	Failed	IN13 332 OUT1 642
✓	OrderSameItemsFromTwoWholeFoodsLocationsBFCL/parallel_54: two whole_foods_order calls — same two items (order-independent) at downtown and uptown, size large.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 200 OUT229
✓	PrimeFactorizeAsDictionaryBFCL/parallel_79: two prime_factorize calls — 4567 and 7890, return_type 'dictionary' (explicitly requested).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 176 OUT880
×	SolveTwoQuadraticEquationsBFCL/parallel_63: two solve_quadratic_equation calls — (5,6,1) and (3,2,1). Prompt names this function explicitly.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 788 OUT865
Tool Calling - Parallel Calls: Mathematics & Physics (set 3) 11 failed1 broken	0.0%	IN608 969 OUT19 402
×	AreaOfTwoCirclesInMetersBFCL/parallel_67: two geometry_calculate_area_circle calls — radii 5 and 10 in meters. unit required (tool default is 'cm'; the prompt states meters).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN17 059 OUT628
!	AreaOfTwoTrianglesBFCL/parallel_69: two calculate_triangle_area calls — base/height 10x5 and 8x6.1 AI query 55 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.Completed Values: response.Completed = false	Broken	IN308 481 OUT6 258
×	DerivativeOfTwoFunctionsAtValuesBFCL/parallel_73: two calculus_derivative calls — 3x^2+2x-1 at 5 (variable x, default) and 4y^3-3y^2+2y-1 at 3 (variable y).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 608 OUT537
×	DisplacementForThreeCarsBFCL/parallel_84: three calculate_displacement calls — (15,7,accel 3.5), (20,10,accel 2.0), (25,8,no acceleration). The third car travels "without any acceleration" so acceleration == 0 (also the default; omission accepted).1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3	Failed	IN24 091 OUT1 281
×	EuclideanNormForThreePointsBFCL/parallel_61: three math_hypot calls — (3,4) and (6,8) with z omitted/0, and (9,12,15).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN18 101 OUT869
×	FactorialOfThreeNumbersBFCL/parallel_60: three math_factorial calls — 5, 7, 9.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN12 647 OUT506
×	FinalSpeedForTwoFreeFallObjectsBFCL/parallel_85: two calculate_final_speed calls — initial_speed 0 for 10 s and initial_speed 5 for 7 s; gravity -9.81 (default, may be omitted).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 611 OUT649
×	GcdViaNumberTheoryBFCL/parallel_78: two number_theory_gcd calls — (45,60) and (81,63). Prompt names this function explicitly.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN22 045 OUT750
×	IntegrateTwoFunctionsWithTwoMethodsBFCL/parallel_72: four integrate calls — x3 on [2,5] and 2x2+3x-1 on [-1,3], each with trapezoid (default, may be omitted) and simpson (explicit).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN13 622 OUT1 616
×	PrimeFactorsOfThreeNumbersBFCL/parallel_75: three number_analysis_prime_factors calls — 45, 100, 150.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3	Failed	IN40 643 OUT1 168
×	SolveFourQuadraticsWithRootTypeBFCL/parallel_64: four solve_quadratic calls — (2,5,3) all roots, (1,-3,2) real, (4,-7,3) all roots, (1,2,1) real. 'all' is the default so it may be omitted; 'real' is explicitly requested and required.1 AI query 40 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 40 expectedCount = 4	Failed	IN93 705 OUT4 199
×	VelocityForTwoCyclingDaysBFCL/parallel_82: two calculate_velocity calls — 120 km in 5 h and 150 km in 6 h; unit km/h (default, may be omitted).1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 2	Failed	IN33 356 OUT941
Tool Calling - Parallel Calls: Mixed Domains 3 passed7 failed	30.0%	IN123 762 OUT6 687
×	CalculateBmiForTwoPeopleBFCL/parallel_4: two calculate_bmi calls — 6 ft / 80 kg and 5.6 ft / 60 kg.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN11 276 OUT305
×	CalculateEmForceForTwoTimeIntervalsBFCL/parallel_1: two calculate_em_force calls — same field/area, d_time 4 then 10.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN11 330 OUT767
✓	CalculateFactorialForThreeNumbersBFCL/parallel_7: three math_factorial calls — 5, 10, 15.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 342 OUT192
×	CalculateResistanceForCopperAndAluminumBFCL/parallel_2: two calculate_resistance calls — copper then aluminum, same length/area. Tightened per methodology: the prompt explicitly names "resistivity of copper", so the first call must pass resistivity == "copper" (ground truth also accepts omission via the 'copper' default, but an explicitly-stated value must be supplied). Aluminum is non-default → required.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN11 324 OUT749
×	CalculateSalesTaxForThreeCitiesBFCL/parallel_6: three calculate_sales_tax calls — Chicago/IL $30.45, Sacramento/CA $52.33, Portland/OR $11.23.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN16 645 OUT1 076
×	FetchPopulationForCitiesStateAndCountryBFCL/parallel_8: four database_us_census_get_population calls — NYC (city), LA (city), Alaska (state), USA (country); year defaults to 2000 (may be omitted). Requires area-type inference.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN12 165 OUT732
×	FindMovieShowingsForTwoMoviesBFCL/parallel_9: two find_movie_showing calls near San Diego — Tenet at 5 pm and No Time To Die at 7:30 pm.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 2	Failed	IN23 218 OUT663
✓	GetSequenceAnd3DForThreeProteinsBFCL/parallel_3: three protein_info_get_sequence_and_3D calls — HbA1c, normal + rat hemoglobin. Tightened per methodology: the prompt explicitly asks for "their 3D models", so model_3d == true is required (ground truth also accepts omission via the default of true).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 421 OUT1 385
×	ListShowRatingsOnNetflixAndHuluBFCL/parallel_5: two streaming_services calls — Netflix['Friends'] and Hulu['The Office','Stranger Things'] (Hulu show order unordered), both sorted by rating. Must group shows per service.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN15 775 OUT672
✓	PlayTwoArtistsForGivenDurationsBFCL/parallel_0: two spotify_play calls — Taylor Swift for 20 min and Maroon 5 for 15 min.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 266 OUT146
Tool Calling - Parallel Calls: Physical & Life Sciences (set 1) 2 passed10 failed	16.7%	IN194 826 OUT12 149
×	ElectricFieldForTwoChargesBFCL/parallel_94: two calculate_electric_field calls — charge 5/distance 2 and charge 3/distance 4. "Conducted in a vacuum" = the default permitivity → may be omitted or 0.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 2	Failed	IN18 889 OUT355
×	ElectromagneticForceInTwoMediaBFCL/parallel_96: two electromagnetic_force calls — same charges 5,10 and distance 2, differing by medium_permittivity. Vacuum call (8.854e-12) equals the default → may be omitted or 8.854e-12; the second medium (5e-12) is non-default → required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 289 OUT1 368
×	EstimateTravelTimeForThreeRoutesBFCL/parallel_93: three route_estimate_time calls — (1) NYC->Miami via Philadelphia/Washington D.C./Atlanta, (2) NYC->Miami skipping Atlanta (via Philadelphia/Washington D.C.), (3) Philadelphia-> Miami via Washington D.C. Stop orderings from ground truth accepted. FLAG parallel_93: BORDERLINE. Ground truth allows many stop-list variants, some including the start city itself as a stop; a competent human's stop set is not fully deterministic. Kept BFCL-faithful by accepting the natural stop orderings (start city excluded from stops).1 AI query 7 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 3	Failed	IN22 167 OUT1 152
✓	FinalSpeedForTwoDroppedObjectsBFCL/parallel_88: two calculate_final_speed calls — v0 0/height 10 and v0 5/height 20. gravity stated as 9.8 (the default) so it may be omitted or 9.8 per ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 420 OUT395
×	FinalVelocityForTwoCarsBFCL/parallel_87: two calculate_final_velocity calls — Car A (v0 0, a 6, t 10), Car B (v0 20, a 4, t 15).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN23 155 OUT772
×	FinalVelocityFromDistanceForTwoObjectsBFCL/parallel_86: two kinematics_final_velocity_from_distance calls — accel 5/dist 100 and accel 10/dist 200. "Both objects start from rest" → initial_velocity may be omitted or 0 (default).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 980 OUT595
×	FindVeganRestaurantsInThreeCitiesBFCL/parallel_91: three vegan_restaurant_find_nearby calls — LA, SF, Seattle, each with operating_hours 22 (prompt: "open until at least 22:00", a non-default value → required).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN18 581 OUT1 030
✓	GenerateFourTravelItinerariesBFCL/parallel_90: four travel_itinerary_generator calls — Tokyo/7/200/urban, Paris/10/150/history, Sydney/5/100/nature, Rome/12/180/culture. Tokyo's "urban" equals the default → omit or "urban".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 791 OUT2 073
×	GetDirectionsForRoundTripBFCL/parallel_89: three get_directions calls — SF->Palo Alto (fastest), Palo Alto->Golden Gate Bridge (scenic), Golden Gate Bridge->SF (fastest). route_type explicitly stated each time.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN18 833 OUT1 042
×	MagneticFieldForTwoLoopsBFCL/parallel_95: two calculate_magnetic_field calls — current 10/radius 0.5 and current 15/radius 1.0. "Same as in free space" = default permeability → may be omitted or 0.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN23 230 OUT1 085
×	ResonantFrequencyForTwoCircuitsBFCL/parallel_97: two calculate_resonant_frequency calls — L 0.005/C 1e-07/round 3 and L 0.007/C 2e-07/round 4. Rounding is stated explicitly → required.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 157 OUT1 424
×	ShortestDrivingDistanceRoundTripBFCL/parallel_92: three get_shortest_driving_distance calls — NYC->LA, LA->Miami, Miami->NYC, all in miles (prompt explicitly "in miles" → non-default unit required).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN13 334 OUT858
Tool Calling - Parallel Calls: Physical & Life Sciences (set 2) 7 passed6 failed	53.8%	IN178 472 OUT13 228
✓	AbsolutePressureAtTwoAltitudesBFCL/parallel_102: two calc_absolute_pressure calls — gauge 2.5 with atm 1 (default; may be omitted) and gauge 2.5 with atm 0.85 (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 525 OUT2 954
×	BoilingMeltingPointsAcrossSeaLevelsBFCL/parallel_100: four get_boiling_melting_points calls — water@0, iron@1000, water@1000, iron@0. FLAG parallel_100: BORDERLINE. The prompt ("water and iron at sea levels of 0 and 1000 meters respectively") reads most naturally as TWO calls (water@0, iron@1000), but the BFCL ground truth is the full 2x2 cross product (FOUR calls). Kept BFCL-faithful to the ground truth; a competent human could reasonably answer with two calls, so this item's expected call count is not fully deterministic.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 986 OUT715
✓	CellInfoDetailedAndBriefBFCL/parallel_108: two biology_get_cell_info calls — neuron with detailed=true (prompt: "detailed description"), muscle with detailed=false ("less detailed", the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 412 OUT868
✓	DensityForTwoSubstancesBFCL/parallel_101: two calculate_density calls (mass/volume schema) — 10 kg / 2 m³ and 15 kg / 3 m³. unit "kg/m³" equals the default → may be omitted or "kg/m³".1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 540 OUT237
✓	ElectricFieldStrengthAtSeveralDistancesBFCL/parallel_98: four calculate_electric_field_strength calls — charge 2, distances 0.5/1/2 in vacuum (default medium), plus distance 1 in air. The distance-1 vacuum and distance-1 air calls are distinguished by medium. Vacuum equals the default → may be omitted or "vacuum"; "air" is required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 835 OUT1 266
×	EntropyChangeForTwoConditionsBFCL/parallel_103: two entropy_change_calculate calls — substance A, mass 2, temps 25->75 and 10->50. pressure 1 equals the default → may be omitted or 1.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 2	Failed	IN27 722 OUT1 282
×	FetchDnaSequenceThreeWaysBFCL/parallel_106: three fetch_DNA_sequence calls for id XYZ123 — default fasta/0, then genbank format, then 500 bp upstream. Defaults (fasta, upstream 0) may be omitted.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN18 831 OUT631
×	FunctionLookupForTwoMoleculesBFCL/parallel_110: two cell_biology_function_lookup calls — ATP in mitochondria and DNA in nucleus, both with specific_function=true (prompt asks for a specific function within the organelle).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN18 055 OUT1 007
✓	HeatCapacityForThreeConditionsBFCL/parallel_105: three calc_heat_capacity calls — 300K/2.5m³, 350K/2.5m³, 300K/1.5m³, all for air. "air" equals the default gas → may be omitted or "air".1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 721 OUT340
×	MolecularWeightForTwoCompoundsBFCL/parallel_111: two calculate_molecular_weight calls — Glucose (C6H12O6) and Sucrose (C12H22O11), both in grams/mole (prompt explicitly requests that unit; ground truth also accepts "g/mol").1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 097 OUT923
×	PhaseChangeEnergyForWaterBFCL/parallel_99: four thermo_calculate_energy calls — 500 g water, phase melting/freezing/ vaporization/condensation. Substance "water" equals the default → may be omitted or "water".1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 4	Failed	IN27 469 OUT1 481
✓	ProteinSequencesForTwoGenesTwoSpeciesBFCL/parallel_107: four get_protein_sequence calls — BRCA1 and BRCA2 in Homo sapiens (default species, may be omitted) and in Pan troglodytes (non-default → required).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 712 OUT744
✓	ProteinsInThreeCompartmentsBFCL/parallel_109: three cellbio_get_proteins calls — nucleus/mitochondria/cytoplasm, each with include_description=true (prompt asks for "a brief description of each protein").1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 567 OUT780
Tool Calling - Parallel Calls: Physical & Life Sciences (set 3) 5 passed7 failed1 broken	38.5%	IN165 366 OUT21 526
×	AnalyzeDnaForThreeMutationTypesBFCL/parallel_114: six analyze_dna_sequence calls — two sequences ("AGCTTAGCTA" and "AGCTTAGGCTA") against reference "AGCTTAGCTA", each checked for insertion (default, may be omitted), deletion, and substitution mutations.1 AI query 7 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 6	Failed	IN22 851 OUT1 136
×	DiabetesPredictionForFourPeopleBFCL/parallel_113: four diabetes_prediction calls for four people with distinct weight/height/activity.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN14 306 OUT1 457
×	EntropyChangeIsothermalAndNotBFCL/parallel_104: two calculate_entropy_change calls — same 300->350 K and heat_capacity 4.18, once isothermal (the default true, may be omitted) and once not isothermal (false, required).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 370 OUT1 358
×	ForestGrowthForTwoRegionsBFCL/parallel_120: two forest_growth_forecast calls — Amazon Rainforest over 10 years and Boreal Forests of Canada over 20 years, excluding human impact (false = the default, may be omitted).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 175 OUT968
!	GenerateFourDnaSequencesBFCL/parallel_123: four generate_DNA_sequence calls — length 500 with a single-nucleotide preference of A, T, C, G respectively.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.Completed Values: response.Completed = false	Broken	IN8 826 OUT10 200
✓	GeneticSimilarityInTwoFormatsBFCL/parallel_115: four genetics_calculate_similarity calls — human vs chimpanzee and human vs gorilla, each in percentage (default, may be omitted) and fraction. species1 accepts "human"/"Human".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 814 OUT617
×	GenotypeFrequenciesForThreeGenotypesBFCL/parallel_116: three calculate_genotype_frequency calls — allele frequency 0.7, genotypes AA/Aa/aa.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN13 613 OUT711
✓	IdentifyFourBirdsBFCL/parallel_119: four identify_bird calls — small blue/forest (default size, may be omitted), large black/lake, medium brown/desert, large green/tropical rainforest.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 922 OUT1 707
✓	MutationTypeForTwoSnpsBFCL/parallel_112: two mutation_type_find calls — rs123456 in Homo sapiens (default species, may be omitted) and rs7891011 in Canis lupus familiaris (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 541 OUT593
×	PopulationDensityForChinaTwoYearsBFCL/parallel_117: two calculate_density calls (country/population schema) — China in 2000 (pop 1.267 billion) and 2010 (pop 1.341 billion), land area 9.597 million km² both times.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN22 409 OUT839
✓	PrecipitationStatsForThreeTimeFramesBFCL/parallel_118: three ecology_data_precipitation_stats calls — Amazon rainforest for six_months, year, five_years.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 614 OUT678
×	TurtlePopulationForTwoYearsBFCL/parallel_121: two ecology_get_turtle_population calls — Galapagos Islands in 2015 and 2020, both with species info requested (species=true; years stated explicitly → required).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 199 OUT721
✓	VehicleEmissionsForThreeVehiclesBFCL/parallel_122: three calculate_vehicle_emission calls — 15000 miles each: gas with default factor 1.4 (may be omitted), diesel with factor 2.7, EV with factor 0.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 726 OUT541
Tool Calling - Parallel Calls: Services & Lookups 2 passed10 failed	16.7%	IN220 494 OUT9 521
✓	CalculateShortestAndScenicDistancesForTwoLegsBFCL/parallel_190: four calculate_shortest_distance calls — {NYC->LA, LA->Miami} x {Shortest, Scenic}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 372 OUT717
×	FetchNewsForTwoTopicsInEuropeBFCL/parallel_192: two get_news calls — Climate Change and Artificial Intelligence, 5 articles each, region Europe. Region "Europe" explicitly requested -> required (accept "Europe"/"EU").1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 458 OUT637
×	FindLibrariesWithFacilitiesInThreeCitiesBFCL/parallel_191: three public_library_find_nearby calls — NY[Reading Room,Fiction], LA[Wi-Fi,Children Section], Chicago[Cafe,Reading Room]. Facilities compared as an unordered pair.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN18 251 OUT918
×	FindParksWithAmenitiesInThreeCitiesBFCL/parallel_189: three parks_find_nearby calls — NY[Tennis Court,Picnic Area], LA[Playground,Running Track], Chicago[Tennis Court,Playground]. Amenities compared as an unordered pair.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN18 201 OUT888
×	FindUpcomingEventsInThreeCitiesBFCL/parallel_194: three event_finder_find_upcoming calls — jazz/LA(14d), rock/Chicago(10d), classical/Boston(7d). All day windows explicitly stated -> days_ahead required (7 for Boston, which also equals the default).1 AI query 21 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 21 expectedCount = 3	Failed	IN43 983 OUT1 777
×	GetCurrentWeatherForFourCitiesBFCL/parallel_199: four get_current_weather calls — New York, Los Angeles, London, Tokyo, with both temperature and humidity included. Prompt explicitly asks to include both -> flags == true (also the default).1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4	Failed	IN18 961 OUT1 060
×	GetDetailedBriefsForTwoMoviesBFCL/parallel_195: two movie_details_brief calls — Inception and The Dark Knight, extra_info true. Prompt asks for Director/Cast/Awards -> extra_info == true required.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN17 067 OUT658
×	GetLawsuitDetailsForTwoCasesBFCL/parallel_196: two get_lawsuit_details calls — case 12345 (NY Supreme Court, with verdict) and case 67890 (LA Superior Court, no verdict). "would like the verdict" -> true; "do not need" -> null/false.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN22 655 OUT763
×	GetLawsuitInfoForTwoCasesByYearBFCL/parallel_197: two lawsuit_info calls — 12345ABC (2018, New York) and 67890XYZ (2019, California). Years are explicitly stated -> required. First location "New York" may be omitted (matches the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 646 OUT471
×	GetStockPricesForTwoCompaniesOverDaysBFCL/parallel_179: two get_stock_price (by-days variant) calls — Apple on NASDAQ for 10 days, Microsoft on NYSE for 15 days. Both exchanges are explicitly named -> required (NASDAQ / NYSE). The tool's 'company' field is a ticker symbol; company names are accepted as synonyms.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN17 181 OUT495
×	SearchLawsuitsForTwoEntitiesInCaliforniaBFCL/parallel_198: two lawsuit_search calls — Google/Santa Clara and Facebook/San Mateo, state California. Prompt states "both in the state of California" -> state required (accept "California"/"CA").1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN22 342 OUT833
✓	SendTwoEmailsWithCcAndBccBFCL/parallel_193: two send_email calls — Project Update to john.doe (cc manager, bcc hr) and Meeting Reminder to jane.doe (cc assistant, bcc hr). Bodies matched exactly.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 377 OUT304
Tool Calling - Parallel Calls: Statistics & Finance 3 passed10 failed	23.1%	IN305 753 OUT15 941
✓	CalculateBinomialProbabilitiesForBasketballShotsBFCL/parallel_163: three calc_binomial_prob calls — (10,6,0.6), (10,6,0.5), (15,6,0.5).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 161 OUT1 186
×	CalculateCompoundInterestForThreePeriodsBFCL/parallel_170: three calculate_compound_interest calls — principal 5000, rate 0.025, time 2/3/5. Prompt states "compounded quarterly, 4 times in a year" -> n == 4 required (tightened; also ground truth).1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3	Failed	IN43 422 OUT1 675
×	CalculateDiscountedCashFlowForTwoBondsBFCL/parallel_169: two calculate_discounted_cash_flow calls — (50,5,0.05) and (60,7,0.04). Prompt explicitly states "face value of the bond is $1000" for both -> face_value == 1000 required (ground truth also accepts omission via the default, but an explicitly-stated value must be supplied).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN19 205 OUT1 525
×	CalculateEventProbabilitiesForCardDeckBFCL/parallel_162: three probability_of_event calls — Ace 4/52 as decimal, heart 13/52 as decimal, red card 26/52 as ratio. format_as_ratio: "as a decimal" -> false or omitted; "as a ratio" -> true (required).1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 3	Failed	IN23 631 OUT785
×	CalculateQuarterlyDividendForThreeScenariosBFCL/parallel_168: three finance_calculate_quarterly_dividend_per_share calls — (5M,2M), (6M,2M), (6M,2.5M).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN20 201 OUT1 094
×	CalculateReturnOnEquityForTwoCompaniesBFCL/parallel_171: two calculate_return_on_equity calls — (1M,5M,200k) and (2M,10M, no dividends). Second company "did not pay any dividends" -> dividends_paid null or 0.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2	Failed	IN30 625 OUT1 161
×	CalculateRoiForThreeStocksBFCL/parallel_174: three calculate_return_on_investment calls — (150,180,20), (200,210,30), (250,300,40). Dividends are explicitly stated per stock -> required.1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 18 expectedCount = 3	Failed	IN39 881 OUT1 701
×	ComputeRSquaredForTwoRegressionModelsBFCL/parallel_167: two linear_regression_get_r_squared calls on the same CSV — swap education/credit_score as dependent variable. Independent-variable order follows the prompt/ground truth.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 2	Failed	IN42 989 OUT1 192
✓	PredictFutureValueForTwoInvestmentsBFCL/parallel_172: two finance_predict_future_value calls — (5000,0.05,10) and (7000,0.04,15). Prompt states "compound interest annually" -> compounding_periods_per_year == 1 required (tightened; also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 968 OUT631
×	PredictHousePricesForThreeHousesBFCL/parallel_166: three predict_house_price calls — NY 2500/3/2000, LA 3000/3/2005, Chicago 2000/2/1995. The second house is described as "similar" with only area/year/location changed, so rooms == 3 per ground truth.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN14 393 OUT945
×	PredictProfitForTwoFundsBFCL/parallel_173: two investment_predictProfit calls — (5000,0.07,5) and (8000,0.05,7).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 778 OUT1 001
×	RunChiSquaredTestForTwoContingencyTablesBFCL/parallel_164: two chi_squared_test calls — tables [45,55,35,65] and [30,70,50,50], alpha 0.05. Prompt states "significance level of 0.05 for both" -> alpha == 0.05 required (tightened). Table order is fixed.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2	Failed	IN30 349 OUT1 628
✓	RunTTestAtTwoSignificanceLevelsBFCL/parallel_165: two t_test calls — same datasets, alpha 0.05 then 0.01 (both stated -> required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 150 OUT1 417
Tool Calling - Simple: Arts, Venues & Music (set 1) 8 failed	0.0%	IN264 402 OUT6 151
×	SimpleConcertDetailsBeyonceBFCL/simple_python_286 (concert.get_details): artist=Beyonce, location=San Diego, date=April 2022.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5	Failed	IN20 822 OUT462
×	SimpleConcertInfoPopNyBFCL/simple_python_284 (concert_info.get): location=New York, date=next month, genre=Pop.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12	Failed	IN51 790 OUT1 109
×	SimpleGenerateMelodyC4BFCL/simple_python_291 (music_generator.generate_melody): key=C, start_note=C4, length=16, tempo=120 or omitted.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7	Failed	IN29 170 OUT730
×	SimpleKeySignatureCsharpBFCL/simple_python_298 (music_theory.key_signature): key='C#', scale_type=major or omitted.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8	Failed	IN32 768 OUT571
×	SimpleMuseumHoursLouvreBFCL/simple_python_273 (museum.get_hours): name='Louvre Museum', location=Paris, day omitted or Monday.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7	Failed	IN28 712 OUT644
×	SimplePaintNeededWall30x12BFCL/simple_python_259 (calculate_paint_needed): coverage_rate=400, length=30, height=12.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6	Failed	IN24 840 OUT576
×	SimpleReligionHistoryChristianity16cBFCL/simple_python_252 (get_religion_history): religion=Christianity, century=16, sort_by=importance, count=10.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6	Failed	IN24 952 OUT1 074
×	SimpleSculptureInfoPlensaBFCL/simple_python_263 (get_sculpture_info): artist_name='James Plensa', detail=true.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12	Failed	IN51 348 OUT985
Tool Calling - Simple: Arts, Venues & Music (set 2) 8 failed	0.0%	IN685 525 OUT11 363
×	SimpleComposeMelodyCFGBFCL/simple_python_292 (compose_melody): progression=[C,F,G], measures=4, instrument=Piano or omitted.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9	Failed	IN37 669 OUT865
×	SimpleConcertFindDetailsWeekndBFCL/simple_python_290 (concert.find_details): artist=The Weeknd, month=December, year=2022 or omitted.1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16	Failed	IN72 942 OUT1 179
×	SimpleFindConcertRockChicagoBFCL/simple_python_285 (find_concert): location=Chicago, price=100, genre=Rock.1 AI query 44 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 44	Failed	IN288 801 OUT3 096
×	SimpleMuseumInfoMetBFCL/simple_python_274 (museum_info): museum_name=Metropolitan Museum of Art, info_type=opening_hours or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5	Failed	IN24 427 OUT414
×	SimpleMusicalScaleCsharpBFCL/simple_python_299 (musical_scale): key='C#'/'C sharp', scale_type=major or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5	Failed	IN21 600 OUT395
×	SimplePaintRequirementWallExclusionBFCL/simple_python_260 (paint_requirement.calculate): area {width=20,height=12}, paint_coverage=350, exclusion {type=window, area=15}.1 AI query 14 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 14	Failed	IN64 821 OUT1 507
×	SimpleReligionHistoryChristianity300To400BFCL/simple_python_254 (get_religion_history): religion=Christianity, start_year=300, end_year=400, event_type=all or omitted.1 AI query 28 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 28	Failed	IN158 500 OUT3 002
×	SimpleSculptureDetailsDavidBFCL/simple_python_264 (sculpture.get_details): artist=Michelangelo, title=David, detail=size.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4	Failed	IN16 765 OUT905
Tool Calling - Simple: Arts, Venues & Music (set 3) 1 passed8 failed	11.1%	IN388 734 OUT7 811
×	SimpleChordProgressionKeyBFCL/simple_python_297 (music.theory.chordProgression): progression=[I,V,vi,IV], bools any.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3	Failed	IN13 915 OUT517
×	SimpleCompoundInterest10kBFCL/simple_python_269 (calculate_compound_interest): principle=10000, interest_rate=0.05, time=10, compounds_per_year=1 or omitted.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6	Failed	IN26 380 OUT777
×	SimpleConcertSearchClassicalLaBFCL/simple_python_287 (concert.search): genre=classical, location=Los Angeles/LA, date='this weekend', price_range=cheap.1 AI query 29 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 29	Failed	IN168 956 OUT2 284
×	SimpleCreateMixCMajorBFCL/simple_python_293 (music_composer.create_mix): scale='C Major', note_duration=quarter, track_length=180.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2	Failed	IN10 262 OUT233
✓	SimpleGenerateCircleImageRedBFCL/simple_python_256 (generate_circle_image): radius=50, color=Red, background=white or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 688 OUT83
×	SimpleMuseumWorkingHoursLouvreBFCL/simple_python_276 (museum_working_hours.get): museum=Louvre, location=Paris, day=Monday or omitted.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7	Failed	IN30 116 OUT707
×	SimpleRetrieveReligionInfoBuddhismBFCL/simple_python_253 (retrieve_religion_info): religion_name=Buddhism, detail_level=full.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11	Failed	IN48 740 OUT1 765
×	SimpleSculptureSearchChicago19cBFCL/simple_python_265 (sculpture_search): location=Chicago, time_frame='19th century', material=all or omitted.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12	Failed	IN53 701 OUT912
×	SimpleTimeDifferenceSfSydneyBFCL/simple_python_250 (get_time_difference): place1=San Francisco/SF, place2=Sydney.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7	Failed	IN29 976 OUT533

Tool Calling - Irrelevance: Architecture & Artwork 8 passed1 failed

88.9%

IN38 267 OUT1 867

✓

NoToolForApartmentRentCostBFCL/irrelevance_158: apartment rental cost asked; only a new-construction cost estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 463 OUT43

×

NoToolForBestJazzSingerBFCL/irrelevance_164: best Jazz singer asked; only a top-instrument-players search (singers are not instrument players) is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2

Failed

IN10 574 OUT502

✓

NoToolForCathedralCeilingDesignBFCL/irrelevance_157: how to design a cathedral ceiling asked; only a lookup of facts about an existing building is available (retrieves data, does not advise on design) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 459 OUT404

✓

NoToolForHolyBookDefinitionBFCL/irrelevance_144: definition of a holy book asked; only a search-within-a-named-holy-book tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 457 OUT103

✓

NoToolForLectureTeacherBFCL/irrelevance_167: teacher of upcoming lectures asked; only an upcoming-concert details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 460 OUT61

✓

NoToolForMostFamousSculptureBFCL/irrelevance_153: most famous sculpture asked; only a sculpting-tools lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 465 OUT176

✓

NoToolForPianoMaintenanceTipsBFCL/irrelevance_166: piano maintenance tips asked; only an instrument rental-prices lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 460 OUT424

✓

NoToolForScreamArtistBFCL/irrelevance_159: artist behind 'The Scream' asked. NOTE: the only tool (artwork_search) requires a museum_location that the user did not provide, so it cannot be correctly called; BFCL labels this irrelevant. Kept faithfully with ground truth = no tool call (flagged as borderline).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 464 OUT18

✓

NoToolForSculptureRockTypeBFCL/irrelevance_152: which rock suits a garden sculpture asked; only a sculpture 3D-model creator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 465 OUT136

Tool Calling - Irrelevance: Buildings, Sculpture & Music 7 passed1 skipped

100.0%

IN23 003 OUT1 377

–

NoToolForCelloInstrumentTypeBFCL/irrelevance_165: instrument classification of a cello asked. NOTE: BFCL labels this irrelevant, but the only tool (get_instrument_info with detail=type) arguably answers it; flagged for review as a borderline BFCL item. Kept faithfully with ground truth = no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_instrument_info' with detail='type' answers 'what type of instrument is a cello' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForDeclarationSignersBFCL/irrelevance_137: who signed the declaration asked; only a historical-event DATE lookup is available (returns a date, not the signatories) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 284 OUT281

✓

NoToolForEiffelTowerHeightBFCL/irrelevance_156: Eiffel Tower height asked; only an architecture-plan generator (creates a new design, does not retrieve facts) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 288 OUT88

✓

NoToolForFamousDancersBFCL/irrelevance_162: famous dancers asked; only a museum-visual-artists lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 290 OUT232

✓

NoToolForReformationInitiatorBFCL/irrelevance_145: who initiated the Reformation asked; only a religious-event YEAR lookup is available (returns a year, not a person) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 283 OUT81

✓

NoToolForScientistDefinitionBFCL/irrelevance_143: definition of "scientist" asked; only a named-historical-figure info lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 280 OUT251

✓

NoToolForSellingGuitarBFCL/irrelevance_163: how to sell a guitar asked; only an instrument-tuning tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 285 OUT301

✓

NoToolForSimilarBlueColorBFCL/irrelevance_148: a color similar to blue asked; only a complementary-color tool (opposite, not similar) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 293 OUT143

Tool Calling - Irrelevance: Cell Biology & Genetics 8 passed2 skipped

100.0%

IN30 331 OUT2 474

✓

NoToolForClottingBloodCellsBFCL/irrelevance_32: blood cell type responsible for clotting asked; only a nucleus-count-based cell-type classifier is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 792 OUT149

✓

NoToolForCommonAncestorBFCL/irrelevance_41: common ancestor of two animals asked (phylogenetics knowledge); only a fibonacci-sequence calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 791 OUT177

✓

NoToolForEvolutionaryAlgorithmInGameAIBFCL/irrelevance_43: how to apply an Evolutionary Algorithm in game AI asked (advice); only a creature-evolution simulator is available (wrong context).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 792 OUT684

✓

NoToolForGeneVariantFrequencyBFCL/irrelevance_36: gene-variant frequency in a population asked; only a dominant-trait-from-alleles tool is available (the true-match variant-frequency tool is deliberately isolated in A4).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 797 OUT220

–

NoToolForIdentifyingGeneticSequenceBFCL/irrelevance_33: interpret a genetic code sequence asked; only a species-identification-from-sequence tool is available (returns a species, not what the sequence means).0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'identify_species' (identifies a species from a genetic sequence) plausibly answers the query about a DNA sequence, so a competent model may call it - BFCL-mislabeled/borderline-relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForIncreasingDeerPopulationBFCL/irrelevance_39: how-to advice on growing an animal population asked; only a population-growth CALCULATOR (needs numeric rates) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 791 OUT300

✓

NoToolForLionMatingProcessBFCL/irrelevance_35: mating process of a species asked; only a dominant/recessive genetic-traits lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 788 OUT438

✓

NoToolForMitochondriaCountBFCL/irrelevance_29: number/count of mitochondria asked; only a cell-part FUNCTION lookup is available (wrong query direction).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 792 OUT191

✓

NoToolForPandaEvolutionaryHistoryBFCL/irrelevance_42: evolutionary history asked (a narrative fact); only a biodiversity-index calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 788 OUT315

–

NoToolForWhaleGeneSequenceBFCL/irrelevance_44: the actual gene sequence of a species asked; only a gene-sequence GENERATOR (needs numeric mutation rate + duration) is available.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'gene_sequencer' (generate gene sequences for evolutionary changes) directly answers the query - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Irrelevance: Color, Geometry & Population 11 passed

100.0%

IN40 858 OUT1 809

✓

NoToolForColorInComputerVisionBFCL/irrelevance_131: color meaning in computer vision asked; only a color-in-personality-psychology analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 714 OUT269

✓

NoToolForDeclarationSigningDateBFCL/irrelevance_135: signing date asked; only date-arithmetic (add days) and age tools are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 712 OUT52

✓

NoToolForElectricityDiscovererBFCL/irrelevance_140: who discovered electricity asked; only an electric-current calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 708 OUT192

✓

NoToolForHexColorMixBFCL/irrelevance_146: mixing two hex colors asked; only a prophet-details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 724 OUT670

✓

NoToolForIndependenceSigningDateBFCL/irrelevance_138: signing date asked; only an age calculator (needs birthdate + current date) and a date-adder are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 712 OUT54

✓

NoToolForMostVisitedMarketBFCL/irrelevance_161: most visited market asked; only a museum visitation-statistics lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 714 OUT92

✓

NoToolForNbaFinalWinnerBFCL/irrelevance_133: NBA final winner asked; only a historical-battle details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 715 OUT48

✓

NoToolForPicassoPaintingYearBFCL/irrelevance_151: year of a Picasso painting asked; only a sculpture-dimensions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 713 OUT125

✓

NoToolForRelativityAuthorBFCL/irrelevance_142: who proposed special relativity asked; only a chemical-element properties lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 717 OUT57

✓

NoToolForSculptureCreatorBFCL/irrelevance_155: sculpture creator asked; only a max-seating-capacity calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 713 OUT68

✓

NoToolForWhichColorsToMixBFCL/irrelevance_150: which colors to mix for a shade asked; only an RGB-to-Pantone converter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 716 OUT182

Tool Calling - Irrelevance: Cooking, Currency & Markets 6 passed2 failed

75.0%

IN58 360 OUT2 060

×

NoToolForBakerySourdoughBFCL/irrelevance_223: a BAKERY selling sourdough asked; only a grocery-shop product finder available -> no tool fits.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 3

Failed

IN13 499 OUT371

✓

NoToolForChickenRecipeBFCL/irrelevance_211: the RECIPE to cook chicken asked; only a cook-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 243 OUT206

✓

NoToolForChocolateCakeIngredientsBFCL/irrelevance_220: cake INGREDIENTS asked; only a cooking-TEMPERATURE tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 246 OUT205

✓

NoToolForCurrencyToGbpBFCL/irrelevance_233: CURRENCY conversion asked; only a WEATHER tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 248 OUT110

✓

NoToolForHowToBoilAnEggBFCL/irrelevance_212: best WAY to boil an egg asked; only a boiling-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 245 OUT349

✓

NoToolForLasagnaRecipeBFCL/irrelevance_219: the RECIPE for lasagna asked; only a recipe-CALORIE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 243 OUT49

✓

NoToolForLegExercisesBFCL/irrelevance_221: leg EXERCISES asked; only a grocery-list generator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 243 OUT169

×

NoToolForPetStoreBFCL/irrelevance_224: a PET store asked; only a GROCERY-store locator available -> no tool fits.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 6

Failed

IN25 393 OUT601

Tool Calling - Irrelevance: Dates & Elements 9 passed1 failed

90.0%

IN44 185 OUT1 700

✓

NoToolForHydrogenPropertiesBFCL/irrelevance_141: chemical properties of Hydrogen asked; only a scientist-contributions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 628 OUT323

✓

NoToolForLargestPlanetBFCL/irrelevance_139: largest planet asked; only a star-info lookup (needs a star name) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 628 OUT177

✓

NoToolForMostImportantProphetBFCL/irrelevance_147: most important prophet asked; only a two-color mixer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 628 OUT68

×

NoToolForMuseumVisitFrequencyBFCL/irrelevance_160: museum member visit frequency asked; only a single most-frequent-visitor lookup is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2

Failed

IN11 528 OUT493

✓

NoToolForPantoneOfSkyBlueBFCL/irrelevance_149: Pantone code of sky blue asked; only a paint-proportions calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 630 OUT177

✓

NoToolForPersonalityTypeBFCL/irrelevance_130: user asks what personality type they are; only a big-five calculator (which needs the user's rated answers) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 626 OUT83

✓

NoToolForStadiumSeatingCapacityBFCL/irrelevance_154: stadium seating capacity asked; only a sculpture-creator lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 629 OUT102

✓

NoToolForVicePresidentBFCL/irrelevance_136: Vice President asked; only a US-president-by-year lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 628 OUT49

✓

NoToolForWaterlooCasualtiesBFCL/irrelevance_132: casualty count of a battle asked; only a historical-event date lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 630 OUT189

✓

NoToolForWorldCupWinnerBFCL/irrelevance_134: sports World Cup winner asked; only a historical-battle outcome predictor is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 630 OUT39

Tool Calling - Irrelevance: Electromagnetism, Thermo & Data 9 passed1 failed

90.0%

IN41 504 OUT2 209

✓

NoToolForElectronRestMassBFCL/irrelevance_23: mass of an electron asked (a knowledge fact); only magnetic-field / current tools are available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 399 OUT30

✓

NoToolForFlightDurationBFCL/irrelevance_20: flight duration asked; only a currency converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 402 OUT123

✓

NoToolForFreezingPointBFCL/irrelevance_25: freezing point asked; only boiling-point calculators are available (wrong phase transition).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 409 OUT346

×

NoToolForGasAmountGeneratedBFCL/irrelevance_26: amount of gas generated asked; only a gas-PRESSURE calculator is available (wrong quantity).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2

Failed

IN10 861 OUT719

✓

NoToolForGiniCoefficientBFCL/irrelevance_48: Gini coefficient asked; only a GDP-fetch tool is available (different economic indicator).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 402 OUT169

✓

NoToolForHeatingWaterBFCL/irrelevance_27: heat energy to warm water asked. NOTE: the original tool (calculate_heat) is a fair match for this query (only the well-known specific heat is missing), which would make "no call" ambiguous; per human-baseline fairness it is deliberately NOT shown here (kept as a distractor in A3), so no visible tool fits and "no call" is unambiguously correct.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 413 OUT331

✓

NoToolForHexagonSidesBFCL/irrelevance_28: number of sides of a hexagon asked (basic geometry knowledge); only a boiling-point tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 400 OUT8

✓

NoToolForSymbolicMagneticFieldBFCL/irrelevance_22: magnetic field asked with SYMBOLIC values (distance 'r', current 'I'); the two magnetic-field tools require concrete numeric distance + current, so there is nothing to pass -> no fit.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 413 OUT210

✓

NoToolForTimeDifferenceBFCL/irrelevance_49: time-zone difference asked; only a geo-coordinate lookup tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 402 OUT223

✓

NoToolForWorldSeriesWinnerBFCL/irrelevance_53: sports trivia asked; only a SQL database-query runner is available (no database/query provided).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 403 OUT50

Tool Calling - Irrelevance: Entertainment, Sports & Knowledge 9 passed

100.0%

IN31 428 OUT1 055

✓

NoToolForBasketballResultBFCL/irrelevance_84: basketball game result asked; only a stock-price retrieval tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 492 OUT35

✓

NoToolForBestSoccerPlayerBFCL/irrelevance_79: best current soccer player asked; only a stock-market prediction tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 490 OUT150

✓

NoToolForCaliforniaGovernorBFCL/irrelevance_87: current Governor of California asked; only a court-case filing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 488 OUT41

✓

NoToolForCrimeThrillerMovieListBFCL/irrelevance_88: best crime-thriller movies of 2020 asked; only a document forgery detector is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 497 OUT306

✓

NoToolForFifaWorldCupWinnerBFCL/irrelevance_80: FIFA World Cup 2010 winner asked; only a stock-ticker lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 493 OUT13

✓

NoToolForFrenchTranslationBFCL/irrelevance_61: French translation of a word asked; only an SVM-model tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 492 OUT12

✓

NoToolForHorrorMovieListBFCL/irrelevance_77: horror movie recommendations asked; only an investment future-value tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 491 OUT377

✓

NoToolForPresidentialElectionWinnerBFCL/irrelevance_85: 2020 presidential election winner asked; only a criminal-case detail lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 493 OUT27

✓

NoToolForSportsNewsBFCL/irrelevance_76: latest sports news asked; only a portfolio investment-trend analyzer is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 492 OUT94

Tool Calling - Irrelevance: Finance & Economics 8 passed

100.0%

IN26 842 OUT1 543

✓

NoToolForAverageHousePriceBFCL/irrelevance_69: market-wide average house price asked; only a regression predictor (needs a trained model + feature vector, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 348 OUT68

✓

NoToolForCompoundInterestBFCL/irrelevance_70: compound-interest computation asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 374 OUT334

✓

NoToolForGoldPriceBFCL/irrelevance_78: today's gold spot price asked; only a bond-price calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 350 OUT46

✓

NoToolForHorrorMovieCatalogBFCL/irrelevance_81: horror movie recommendations asked; only a historical stock-price fetcher is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 351 OUT359

✓

NoToolForLossProjectionBFCL/irrelevance_73: a loss projection asked; only a revenue-prediction tool is available (revenue != loss).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 353 OUT488

✓

NoToolForPoundToKilogramBFCL/irrelevance_75: a mass unit conversion asked; only a portfolio-returns calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 349 OUT18

✓

NoToolForProfitMarginBFCL/irrelevance_71: profit margin asked; only an ROI calculator is available (ROI != margin) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 371 OUT138

✓

NoToolForStockNewsBFCL/irrelevance_82: stock news asked; only a capital-gains calculator is available (gains != news) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 346 OUT92

Tool Calling - Irrelevance: Finance Near-Matches 7 passed2 failed

77.8%

IN53 049 OUT7 862

✓

NoToolForBitcoinPriceBFCL/irrelevance_56: current bitcoin price asked; only a generic database record-select tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 428 OUT40

✓

NoToolForBookstoreLookupBFCL/irrelevance_55: which online bookstore sells a title asked; only an add-to-cart tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 433 OUT65

×

NoToolForBusinessRateOfReturnBFCL/irrelevance_74: rate of return asked; only a net-profit calculator is available (profit != rate of return).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2

Failed

IN15 169 OUT2 136

✓

NoToolForCoinFlipProbabilityBFCL/irrelevance_66: probability of a coin landing heads asked; only a standard-deviation tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 433 OUT103

✓

NoToolForCurrentInterestRateBFCL/irrelevance_83: the current market interest rate asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 427 OUT65

×

NoToolForExternalRateOfReturnBFCL/irrelevance_72: EXTERNAL rate of return asked; only an internal-rate-of-return (IRR) tool is available (ERR != IRR, even though cash flows are given) -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2

Failed

IN13 847 OUT3 962

✓

NoToolForFinalPriceWithoutBasePriceBFCL/irrelevance_60: final price after discount+tax asked, but the original price is never given, so the (domain-matching) calculateFinalPrice cannot be correctly called -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 447 OUT183

✓

NoToolForFrontendDevelopmentBFCL/irrelevance_62: how to build an e-commerce frontend asked; only a recommender-model builder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 433 OUT1 141

✓

NoToolForHighestGrossingMovieBFCL/irrelevance_54: highest-grossing movie of all time asked; only a movie search (requires a specific title + year, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 432 OUT167

Tool Calling - Irrelevance: Food, Time & Distance 3 passed1 failed1 skipped

75.0%

IN16 440 OUT872

✓

NoToolForCakeBakingMachineBFCL/irrelevance_218: a baking MACHINE/appliance asked; only a recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 708 OUT306

✓

NoToolForDrivingTravelTimeBFCL/irrelevance_215: driving TRAVEL TIME asked; only a restaurant-bill tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 712 OUT129

×

NoToolForLondonTimezoneNameBFCL/irrelevance_226: the NAME of London's timezone asked; only a local-TIME-by-timezone tool available (it needs the timezone as input and returns a time, not the timezone name) -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2

Failed

IN8 304 OUT296

–

NoToolForPizzaInBostonBFCL/irrelevance_213: a place for pizza asked; a cuisine+location restaurant_finder is available. REVIEW FLAG (fairness): restaurant_finder(cuisine=pizza/Italian, location=Boston) plausibly answers this, so a competent human might legitimately call it. BFCL labels it irrelevant; kept as such and marked Hard because the intended "no call" is debatable. No OTHER query here is a restaurant search, so the tool cannot wrongly satisfy the file's other cases.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'restaurant_finder' (cuisine+location) answers 'good place for pizza in Boston' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForTriangleHypotenuseBFCL/irrelevance_238: the HYPOTENUSE of a triangle asked; only a geo lat/long distance calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 716 OUT141

Tool Calling - Irrelevance: Geo, Travel & Sentiment 7 passed1 failed1 skipped

87.5%

IN157 657 OUT3 595

✓

NoToolForAverageTemperatureBmiBFCL/irrelevance_110: an average December temperature asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 398 OUT30

✓

NoToolForCaseLawyerBFCL/irrelevance_95: the lawyer of a case asked; only a case-outcome retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 401 OUT59

✓

NoToolForCurrentTimeSunriseBFCL/irrelevance_102: the current time asked; only a sunrise/sunset time calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 397 OUT32

✓

NoToolForFamousQuoteBFCL/irrelevance_119: a famous quote asked; only a sentiment-analysis tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 395 OUT32

✓

NoToolForFastestRouteBFCL/irrelevance_114: the fastest route asked; only a closest-airport finder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 399 OUT122

–

NoToolForRestaurantsInLondonBFCL/irrelevance_113: restaurants in London asked; only a generic points-of-interest locator is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'find_pois' (locate points of interest by category) answers 'restaurants in London' (restaurants are POIs) - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForSportsBrainImpactBFCL/irrelevance_120: the neurological impact of sports asked; only a caffeine neurological-impact tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 398 OUT248

✓

NoToolForTrafficConditionBFCL/irrelevance_112: the current traffic condition asked; only an address geocoder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 397 OUT49

×

NoToolForTravelTimeByCarBFCL/irrelevance_115: travel time by car asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 23 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 23

Failed

IN133 872 OUT3 023

Tool Calling - Irrelevance: Legal & Case Lookup 9 passed1 failed

90.0%

IN42 912 OUT1 367

✓

NoToolForAppleCaloriesBFCL/irrelevance_100: calories in an apple asked; only a litigation-cost calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 550 OUT39

✓

NoToolForBrainWeightBFCL/irrelevance_122: average human brain weight asked; only a per-patient neural-activity tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 552 OUT67

✓

NoToolForCaseJudgementBFCL/irrelevance_93: a case judgement asked; only a law-firm impactful-cases lister is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 550 OUT80

✓

NoToolForChineseRestaurantBFCL/irrelevance_97: a Chinese-restaurant recommendation asked; only a lawsuit-filing tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 553 OUT188

✓

NoToolForMovieRecommendationBFCL/irrelevance_116: a movie recommendation asked; only a word-count tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 551 OUT156

×

NoToolForNbaChampionBFCL/irrelevance_118: the 1996 NBA champion asked; only an NBA playoff-brackets display tool is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2

Failed

IN10 938 OUT349

✓

NoToolForPaintingEiffelSettlementBFCL/irrelevance_98: how long to paint the Eiffel Tower asked; only a lawsuit-settlement estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 554 OUT106

✓

NoToolForSecurityGuardSearchBFCL/irrelevance_92: finding a security guard asked; only a lawyer-search tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 560 OUT42

✓

NoToolForSphereVolumeBFCL/irrelevance_104: sphere volume asked; only a circular-park area tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 554 OUT216

✓

NoToolForTrafficLawsBFCL/irrelevance_99: traffic laws asked; only a lawsuit-search tool is available (lawsuits, not laws) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 550 OUT124

Tool Calling - Irrelevance: Math & Science Distractors 8 passed

100.0%

IN25 252 OUT1 352

✓

NoToolForDefiniteIntegralBFCL/irrelevance_9: definite integral asked; only an income-tax calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 159 OUT159

✓

NoToolForIntegralOfCubeBFCL/irrelevance_8: definite integral asked; only a string-to-int converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 152 OUT254

✓

NoToolForLatestMoviesBFCL/irrelevance_15: latest movie releases asked; only a physics final-velocity tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 146 OUT68

✓

NoToolForPerpendicularSlopeBFCL/irrelevance_3: slope of a perpendicular line asked; only a critical-points finder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 164 OUT189

✓

NoToolForPrimeFactorsBFCL/irrelevance_13: prime factorization asked; only a compound-interest calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 149 OUT95

✓

NoToolForRectangleAreaBFCL/irrelevance_6: rectangle area asked; only a circle-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 160 OUT125

✓

NoToolForTriangleAreaBFCL/irrelevance_0: triangle area asked; only a BMI tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 160 OUT169

✓

NoToolForUpwardAccelerationBFCL/irrelevance_14: acceleration asked (it is just gravity); the tempting max-height and final-velocity tools compute neither the acceleration nor what is asked -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 162 OUT293

Tool Calling - Irrelevance: Math & Statistics 5 passed1 failed

83.3%

IN21 941 OUT3 620

✓

NoToolForBinomialMeanBFCL/irrelevance_67: the MEAN of a binomial experiment asked; only p-value tools are available (they return a p-value, not the mean) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 028 OUT294

✓

NoToolForCivilLawComponentsBFCL/irrelevance_90: the components/categories of Civil Law asked; only a noise-complaint filing tool is available (filing a complaint does not explain law) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 000 OUT232

✓

NoToolForCoinTossMaxHeadsBFCL/irrelevance_63: how many heads CAN be obtained from tossing 3 coins (a count/maximum, not a probability) asked; only a probability-of-an-event calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 004 OUT106

×

NoToolForNullHypothesisStdDevBFCL/irrelevance_68: the STANDARD DEVIATION of a null-hypothesis test asked; only p-value tools are available (they return a p-value, not a standard deviation) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 897 OUT2 583

✓

NoToolForPowerOfTwoBFCL/irrelevance_59: raising a number to a power asked; only a linear-equation solver is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 003 OUT38

✓

NoToolForQuadraticEquationBFCL/irrelevance_57: solving a quadratic equation asked; only a genetic-algorithm optimizer and a LINEAR equation solver are available (a quadratic is not linear) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 009 OUT367

Tool Calling - Irrelevance: Medical, Social & Personality 6 passed3 skipped

100.0%

IN20 464 OUT936

✓

NoToolForBigMacCaloriesBFCL/irrelevance_123: calories of a Big Mac asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 409 OUT43

✓

NoToolForEconomicHappinessEffectBFCL/irrelevance_126: the effect of economic status on happiness asked; only a happiness-index lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 411 OUT113

✓

NoToolForFriendMbtiBFCL/irrelevance_129: a friend's MBTI asked; only a personality-type evaluator (needs trait percentages) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 409 OUT85

–

NoToolForLatestTechTrendBFCL/irrelevance_124: the latest technology trend asked; only a trending-topics-by-category tool is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_social_trends' (trending topics by category) answers 'latest trend in technology' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForMajorAirportsBFCL/irrelevance_117: major US airports asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 412 OUT224

–

NoToolForMotorNeuronDiseaseBFCL/irrelevance_121: motor neuron disease info asked; only a generic disease-info retriever is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'medical_records_get_disease_info' directly answers 'information on motor neuron diseases' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForPopularBooksBFCL/irrelevance_125: popular books by an author asked; only a recent-tweets retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 411 OUT259

–

NoToolForTwitterMoodBFCL/irrelevance_127: overall twitter mood asked; only a twitter sentiment-analysis tool is available -> BFCL ground truth is no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'sentiment_analysis_twitter' directly answers 'general mood of twitter regarding the new iPhone' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForVegetableServingsBFCL/irrelevance_128: daily vegetable servings asked; only a personality-test scorer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 412 OUT212

Tool Calling - Irrelevance: Mixed Distractors 8 passed

100.0%

IN26 008 OUT962

✓

NoToolForAirQualityBFCL/irrelevance_40: current air quality asked; only a plant-biomass calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 252 OUT125

✓

NoToolForFavoriteRestaurantBFCL/irrelevance_31: a favorite restaurant recommendation asked; only a cell-division simulator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 248 OUT45

✓

NoToolForGlobalWarmingDocumentaryBFCL/irrelevance_38: a documentary (video) asked; only a scientific-research-paper retriever is available (wrong medium).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 249 OUT49

✓

NoToolForLasagnaIngredientsBFCL/irrelevance_47: recipe ingredients asked; only a flight-timetable tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 249 OUT165

✓

NoToolForLionDominantTraitBFCL/irrelevance_34: dominant genetic trait of a species asked; only a gene-variant-FREQUENCY lookup is available (different genetic output).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 251 OUT117

✓

NoToolForMultinucleatedCellNameBFCL/irrelevance_30: name of a cell type with multiple nuclei asked; only a blood-cell classifier (by shape/size) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 255 OUT175

✓

NoToolForPicnicSpotBFCL/irrelevance_37: a picnic spot asked; only a local-fauna information tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 248 OUT221

✓

NoToolForTranslationBFCL/irrelevance_52: natural-language translation asked; only a file-compression tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 256 OUT65

Tool Calling - Irrelevance: Motion, Electricity & Misc 10 passed

100.0%

IN35 428 OUT2 270

✓

NoToolForCarProjectileRangeBFCL/irrelevance_16: projectile-style "how far will a car travel" asked with symbolic v/t/theta; the range tool needs concrete numeric velocity + angle -> nothing to pass -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 553 OUT435

✓

NoToolForConeVolumeBFCL/irrelevance_19: cone volume asked; only an investment-return calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 547 OUT383

✓

NoToolForCurrentClockTimeBFCL/irrelevance_17: current clock time asked; only a travel-time (distance/speed) calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 536 OUT82

✓

NoToolForElectronMassBFCL/irrelevance_24: mass of an electron asked (a knowledge fact); only an electric-current tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 537 OUT25

✓

NoToolForFibonacciOf20BFCL/irrelevance_51: fibonacci number asked; only a cryptocurrency-price tool is available (the fibonacci calculator lives in a different file so it cannot satisfy this query).1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 538 OUT151

✓

NoToolForForceAngleFromMomentumBFCL/irrelevance_18: angle of a force from momentum asked; only a vector-angle-from-XY-components tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 543 OUT290

✓

NoToolForHorrorMovieListBFCL/irrelevance_50: movie recommendations asked; only a Celsius-to-Fahrenheit converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 543 OUT329

✓

NoToolForMagneticFieldNearWireBFCL/irrelevance_21: magnetic field near a current-carrying wire asked; only an EM-wave-amplitude tool is available (magnetic-field calculators are deliberately kept in a different file).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 552 OUT313

✓

NoToolForSineOf45DegreesBFCL/irrelevance_45: sine of an angle asked; only a polygon-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 538 OUT145

✓

NoToolForTeslaPriceBFCL/irrelevance_46: product price of a car asked; only a currency-exchange-rate tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 541 OUT117

Tool Calling - Irrelevance: Music Theory & Lookup 8 passed

100.0%

IN25 588 OUT963

✓

NoToolForAncientGreekLawmakersBFCL/irrelevance_171: who established laws in Ancient Greece asked (history); only a musical-piece composer search is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 196 OUT160

✓

NoToolForBasketballGameAtVenueBFCL/irrelevance_169: who is playing a basketball game at a venue asked; only concert-locating tools exist.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 199 OUT72

✓

NoToolForBluesScalePatternBFCL/irrelevance_176: the interval pattern of a blues scale asked; only composer lookups are available (get_blues_scale lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 201 OUT242

✓

NoToolForDonQuixoteAuthorBFCL/irrelevance_172: author of the novel Don Quixote asked; only music-composition lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 194 OUT16

✓

NoToolForFamousUsComposersBFCL/irrelevance_170: most famous US composers asked (general knowledge, no specific piece); only a chord-progression creator + piece-specific composer lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 197 OUT195

✓

NoToolForGrammyBestAlbumBFCL/irrelevance_177: Grammy Award winner asked; only a song chord-progression lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 202 OUT83

✓

NoToolForPrimaryTriadsOfKeyBFCL/irrelevance_173: primary triads of a key asked; only a genre-common-chords finder is available (primary_triads lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 200 OUT105

✓

NoToolForUniversityClassAvailabilityBFCL/irrelevance_168: availability of a university class asked; only concert-availability / concert-search tools are available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 199 OUT90

Tool Calling - Irrelevance: Probability & Legal 4 passed1 failed

80.0%

IN17 050 OUT2 748

✓

NoToolForFaceCardProbabilityBFCL/irrelevance_64: probability of drawing a face card from a deck asked; only a COIN-toss heads probability tool is available (coins, not cards) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 798 OUT172

×

NoToolForHomeElectricityNeedBFCL/irrelevance_58: home electricity demand (kWh) asked; only a solar-panel COUNT calculator is available (it sizes panels, not electricity need, and needs sunlight hours not given) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN5 844 OUT693

✓

NoToolForNoiseComplaintProcessBFCL/irrelevance_91: how/whether to report a noise complaint asked; only a law-category listing tool is available (it lists legal categories, it does not file or advise on complaints) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 798 OUT67

✓

NoToolForRedMarbleCountBFCL/irrelevance_65: number of red marbles asked; only a determine-population tool is available (it returns a total population from probability + sample size, not a sub-category count) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN2 813 OUT1 485

✓

NoToolForTenantRightsBFCL/irrelevance_89: explaining tenant rights asked; only a contract-generation tool is available (generating a contract does not explain rights) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN2 797 OUT331

Tool Calling - Irrelevance: Recipes & Cooking 8 passed1 skipped

100.0%

IN27 658 OUT1 412

✓

NoToolForBestGameCharacterBFCL/irrelevance_206: best game character vs a dragon asked; only a best-WEAPON finder available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 461 OUT159

✓

NoToolForBestMovie2020BFCL/irrelevance_235: best MOVIE of 2020 asked; only a currency-exchange calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 458 OUT65

✓

NoToolForBuildingAPcBFCL/irrelevance_209: how to build a PC asked; only a cooking-recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 453 OUT797

✓

NoToolForFamousPlaceInParisBFCL/irrelevance_210: most famous PLACE in Paris asked; only a restaurant-by-dish search available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 455 OUT68

✓

NoToolForGameShortestPathBFCL/irrelevance_208: shortest in-game path asked; only a taxi-fare calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 463 OUT64

✓

NoToolForInchToMeterBFCL/irrelevance_234: LENGTH conversion (inches to metres) asked; only a CURRENCY converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 454 OUT66

✓

NoToolForLowestGameScoreBFCL/irrelevance_207: LOWEST game score asked; only a HIGHEST-score retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 458 OUT95

✓

NoToolForNewYorkPopulationBFCL/irrelevance_225: current POPULATION of New York asked; only a time-zone converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 456 OUT98

–

NoToolForQuickestFlightLondonTokyoBFCL/irrelevance_236: quickest way to fly London->Tokyo asked; a flight-duration tool is available. REVIEW FLAG (fairness): get_flight_duration plausibly answers this ("quickest ... by plane" ~= quickest flight duration), so a competent human might legitimately call it. Kept as BFCL labels it (irrelevant), but this item is borderline. Marked Hard because the intended "no call" is debatable.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_flight_duration' (with flight_type non-stop) plausibly answers 'quickest way ... by plane' - BFCL-mislabeled/borderline-relevant. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Irrelevance: Restaurants & Travel 7 passed1 failed

87.5%

IN50 172 OUT1 040

✓

NoToolForBeijingSunsetBFCL/irrelevance_227: SUNSET time asked; only a SUNRISE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 315 OUT43

✓

NoToolForClosestItalianRestaurantBFCL/irrelevance_216: FIND the closest Italian restaurant asked; only a currency converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 314 OUT53

✓

NoToolForFindSushiRestaurantBFCL/irrelevance_214: FIND a sushi restaurant asked; only a tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 316 OUT54

✓

NoToolForNearestPharmacyBFCL/irrelevance_237: FIND the nearest pharmacy asked; only a route-to-a-known-location planner available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 316 OUT59

✓

NoToolForPetFriendlyTrainStationBFCL/irrelevance_230: a pet-friendly TRAIN STATION asked; only hotel search tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 315 OUT54

✓

NoToolForSushiRestaurantsTokyoBFCL/irrelevance_229: FIND sushi RESTAURANTS asked; only hotel-booking tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 316 OUT189

✓

NoToolForThaiRestaurantChicagoBFCL/irrelevance_231: FIND a Thai RESTAURANT asked; only a hotel-availability checker available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 317 OUT52

×

NoToolForYogaRetreatBFCL/irrelevance_232: a yoga RETREAT asked; only a HOTEL search tool available. REVIEW FLAG (fairness): a retreat is not a hotel, but hotel_search_find_hotels accepts arbitrary amenities (spa, vegetarian) and a location, so it is somewhat tempting. BFCL labels this irrelevant (retreat != hotel); kept as such and marked Hard because the boundary is borderline.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 6

Failed

IN26 963 OUT536

Tool Calling - Irrelevance: Sports & Games (set 1) 9 passed1 failed

90.0%

IN43 035 OUT1 632

×

NoToolForAssistAverageBFCL/irrelevance_185: assist average asked; only an average-SCORING tool is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2

Failed

IN10 899 OUT189

✓

NoToolForBestChessMoveBFCL/irrelevance_194: best next move asked; only a board-game-STATISTICS tool is available (the move-suggesting board_analyser lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 574 OUT88

✓

NoToolForChessArrangementCountBFCL/irrelevance_196: total theoretical arrangements asked; only a final-score calculator is available (possible_moves lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 574 OUT160

✓

NoToolForFootballTeamRankingBFCL/irrelevance_186: ranking of a team asked; only a season-MVP tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 570 OUT79

✓

NoToolForIceCreamStoreBFCL/irrelevance_201: find a physical store asked; only a video-game guide is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 567 OUT77

✓

NoToolForMostHeartsInPokerBFCL/irrelevance_199: who holds most of a suit asked; only a poker-dealing tool is available (get_highest_card_holder lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 574 OUT174

✓

NoToolForPlayerBiographyBFCL/irrelevance_189: a biographical "who is X" asked; only a top-ranked-athlete lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 567 OUT84

✓

NoToolForTeamRosterBFCL/irrelevance_192: full roster asked; only a top-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 569 OUT83

✓

NoToolForUnoRulesBFCL/irrelevance_198: the rules of a game asked; only a deck-shuffling tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 571 OUT627

✓

NoToolForWorldCupWinnerBFCL/irrelevance_191: who won a tournament asked (no team named); only a per-team match-stats tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 570 OUT71

Tool Calling - Irrelevance: Sports & Games (set 2) 8 passed1 failed

88.9%

IN38 324 OUT927

✓

NoToolForBlackjackAceRuleBFCL/irrelevance_200: a rules question asked; only a poker highest-card-holder tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 469 OUT123

✓

NoToolForBookAuthorBFCL/irrelevance_205: author of a book asked; only a video-game score calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 473 OUT19

✓

NoToolForChessTournamentWinnerBFCL/irrelevance_195: who won a tournament asked; only a board-position move-suggester is available (the game-history fetch_game_stats lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 470 OUT79

✓

NoToolForCreateJigsawBFCL/irrelevance_204: CREATE a puzzle asked; only a puzzle-SOLVING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 463 OUT124

×

NoToolForHaloPlayerRankBFCL/irrelevance_203: a player's RANK asked; only a player-SCORE lookup is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2

Failed

IN10 581 OUT195

✓

NoToolForLakersTopScorerBFCL/irrelevance_193: top scorer of a team asked; only a team-details tool (roster/results/upcoming, not scorers) is available (sports_team.get_top_scorer lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 468 OUT162

✓

NoToolForMonopolyWinnerBFCL/irrelevance_197: who won a casual game asked; only a possible-moves calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 468 OUT79

✓

NoToolForTopTennisPlayerBFCL/irrelevance_190: top ranked PLAYER asked; only a TEAM standing/ranking tool is available (the athlete-ranking get_top_ranked lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 467 OUT35

✓

NoToolForWorldSeriesWinnerBFCL/irrelevance_202: who won a real game asked; only a score calculator (from raw points) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 465 OUT111

Tool Calling - Irrelevance: Sports Statistics 8 passed1 failed

88.9%

IN42 684 OUT1 132

✓

NoToolForBasketballSeasonMvpBFCL/irrelevance_187: who won the MVP asked; only a team-ranking tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 511 OUT161

✓

NoToolForClippersRosterBFCL/irrelevance_179: roster of a team asked; only a two-team game-results tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 504 OUT60

✓

NoToolForCommonChordsInPopBFCL/irrelevance_174: most common chords in a pop song asked; only a key-signature primary-triads tool is available (find_common_chords lives in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 509 OUT198

×

NoToolForCricketMatchParticipantsBFCL/irrelevance_180: who is playing in cricket matches today asked; only a cricket-schedule tool (dates/matches, not participants) is available -> no tool fits.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 3

Failed

IN14 614 OUT295

✓

NoToolForLaLigaPlayersBFCL/irrelevance_181: who played in a whole league asked; only a two-named-team last-match-result tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 504 OUT182

✓

NoToolForMoonlightSonataComposerBFCL/irrelevance_175: composer of Moonlight Sonata asked; only a blues-scale generator is available here (all composer-lookup tools are isolated in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 507 OUT24

✓

NoToolForNbaMvpWinnerBFCL/irrelevance_184: who won an MVP award asked (no player named); only a per-player stats tool that requires a player name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 513 OUT103

✓

NoToolForTopAssistPlayerBFCL/irrelevance_178: top ASSIST player asked; only a top-GOAL-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 508 OUT49

✓

NoToolForWimbledonChampionBFCL/irrelevance_183: winner of a tournament asked (no person named); only a celebrity-info tool that requires a name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 514 OUT60

Tool Calling - Irrelevance: Time, Weather & Misc 3 passed1 skipped

100.0%

IN7 742 OUT338

✓

NoToolForCityDistanceKmBFCL/irrelevance_239: DISTANCE (km) between two cities asked; only a get_date tool that returns the TIME difference between two locations is available (its 'unit' arg mentions distance, but it computes time) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 584 OUT87

–

NoToolForCurrentTimeSydneyBFCL/irrelevance_228: current time in Sydney asked; a local-time-by-city tool is available. REVIEW FLAG (fairness): get_local_time(location='Sydney') directly answers this, so a competent human would legitimately call it. BFCL nonetheless files it under irrelevance; kept as such to preserve the source label, but this item is effectively RELEVANT and should be reviewed. Marked Hard because the "no call" expectation is contradicted by the tool itself.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_local_time' directly answers 'current time in Sydney' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForTomatoCaloriesBFCL/irrelevance_222: CALORIES in a tomato asked; only a grocery-item-details lookup available. REVIEW FLAG (fairness): item_details at 'detailed' level could arguably surface nutrition info, and it requires a store_location the user never gave; BFCL labels it irrelevant (it is a grocery-store item lookup, not a nutrition-facts service). Kept as such; marked Medium as borderline.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 580 OUT87

✓

NoToolForWriteABookBFCL/irrelevance_217: "can you write a book" asked; only a recipe-CREATION tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN2 578 OUT164

Tool Calling - Irrelevance: Weather, Environment & Services 9 passed1 failed

90.0%

IN75 987 OUT2 735

✓

NoToolForBestCo2ReductionBFCL/irrelevance_108: the best way to reduce CO2 asked; only a numeric emission-reduction estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 662 OUT245

✓

NoToolForBestMonthToVisitBFCL/irrelevance_101: best month to visit Hawaii asked; only an average-monthly-temperature tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 660 OUT224

✓

NoToolForCactusNutrientsBFCL/irrelevance_109: a plant's nutrient needs asked; only a weekly watering-needs calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 667 OUT497

✓

NoToolForCurrentTimeWeatherBFCL/irrelevance_103: the current time asked; only a current-weather forecast tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 661 OUT140

✓

NoToolForGardenSpeciesCompositionBFCL/irrelevance_107: garden species composition asked; only a soil sand/silt/clay composition analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 664 OUT300

×

NoToolForGmoYogaMatBFCL/irrelevance_111: a GMO yoga mat asked; only an eco-friendly product locator is available (GMO is not eco-friendly) -> no tool fits.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 9

Failed

IN43 013 OUT839

✓

NoToolForImpactfulCasesBFCL/irrelevance_94: a firm's impactful cases asked; only a single-case detail lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 671 OUT75

✓

NoToolForPaintingEiffelCaseResultBFCL/irrelevance_96: how long to paint the Eiffel Tower asked; only a law-case result lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 663 OUT109

✓

NoToolForPollinationFlowersBFCL/irrelevance_105: top pollination flower species asked; only a route elevation-profile plotter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 665 OUT176

✓

NoToolForTomatoFertilizerBFCL/irrelevance_106: best fertilizer for tomatoes asked; only a soil-type analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 661 OUT130

Tool Calling - Parallel Calls: Art, Music & Local Services 1 passed9 failed1 skipped

10.0%

IN213 221 OUT8 855

–

CalculateDrivingDistancesShortestRouteBFCL/parallel_38: two calc_distance calls — New York→Washington DC and Los Angeles→San Francisco. Tightened per methodology: the prompt explicitly enables "shortest route", so shortest_route == true is required (schema default is false).0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Skipped: Ambiguous scope (item flaw, not model behavior): the trailing 'with optional parameter shortest route enabled' can reasonably apply to only the second leg (its nearest clause) or to both; the BFCL ground truth requires it on both, but a competent human would not answer deterministically. Kept for traceability.

Skipped

IN0 OUT0

×

CheckInstrumentAvailabilityInTwoCitiesBFCL/parallel_42: two check_instrument_availability calls — Yamaha P125 in Berlin and Madrid.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN16 025 OUT580

×

FindMusicStoresInTwoCitiesBFCL/parallel_41: two find_music_instrument_store calls — San Francisco and Chicago; instruments {Yamaha acoustic guitar, Kawai piano} in any order and either casing.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN20 955 OUT858

×

FindRockAndJazzConcertsInTwoCitiesBFCL/parallel_43: four concert_finder calls — {San Francisco, New York} x {rock, jazz} over the next month. time_period may be omitted (defaults to 30 days ≈ "next month"). Isolated from concert_find_nearby (parallel_44) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN12 560 OUT654

×

GetExhibitionDetailsForTwoMuseumsBFCL/parallel_40: two museum_exhibition_detail calls — 'Wonder of Nature' at the Louvre and 'Age of Reptiles' at the British Museum. visitor_type {child, adult} (unordered). Tightened per methodology: the prompt explicitly asks for cost "for children and adult", so both visitor types are required (schema default is ['adult'] only).1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2

Failed

IN32 191 OUT1 262

×

GetMostPlayedPopAndRockSongsBFCL/parallel_45: two musicCharts_getMostPlayed calls — most played Pop and Rock song in Australia. Optional duration omitted (ground truth accepts the schema default of 0).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN11 567 OUT458

×

GetMuseumInfoForTwoCitiesBFCL/parallel_39: two museum_info_get_info calls — Washington D.C. and Paris; details {Opening hours, Adult tickets, Child tickets} in any order.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN20 903 OUT894

×

GetRgbAndHexForThreeColorsBFCL/parallel_37: three color_converter_get_color_info calls — navy, purple, maroon; each with {RGB, HEX} (unordered).1 AI query 21 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 21 expectedCount = 3

Failed

IN41 103 OUT1 790

×

GetTeamRankingsInTwoLeaguesBFCL/parallel_47: two get_team_ranking calls — Barcelona in UEFA Champions League and Manchester United in La Liga.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN11 573 OUT292

×

MixPaintColorsForTwoRequestsBFCL/parallel_36: two paint_color_mixture calls — watercolor magenta and acrylic navy.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 11 expectedCount = 2

Failed

IN38 574 OUT1 571

✓

WinningPercentageForTwoTeamsTwoSeasonsBFCL/parallel_46: four calculate_winning_percentage calls — {Lakers, Bulls} x {2018, 2020}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 770 OUT496

Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 1) 3 passed10 failed

23.1%

IN222 611 OUT11 211

✓

AverageFourNumberSetsBFCL/parallel_132: four calculate_average calls — one per list of numbers (element order preserved).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 768 OUT335

×

CalculateBmiFourPeopleMetricBFCL/parallel_134: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt states "All measurements are in the metric system", so unit == "metric" is required (also the default).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN14 033 OUT905

×

CalculateCityDistancesInKilometersBFCL/parallel_135: three geo_distance_calculate calls — NY->LA->Miami->NY, in kilometers. Tightened: the tool defaults to miles but the prompt asks "in kilometers", so units == "kilometers" is REQUIRED (omission would return miles, contradicting the request — the ground truth's "omit" option is unsafe here).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 3

Failed

IN13 099 OUT834

×

CalculateGpsDistancesAlongRouteBFCL/parallel_133: three calculate_distance calls over GPS coordinate pairs (Paris->Rome->Athens-> Giza), each in kilometers. Uses the coordinate-based tool (coord1/coord2), not the city-name distance tools that are also visible.1 AI query 13 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 13 expectedCount = 3

Failed

IN54 665 OUT1 462

×

CompareAverageTemperatureNyAndLaBFCL/parallel_128: two average_temperature calls — New York 7 days in Fahrenheit, Los Angeles 7 days in Celsius. Tightened: the prompt explicitly requests Fahrenheit for NY, so temp_unit == "Fahrenheit" is required (also the default); Celsius is non-default and required for LA.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 926 OUT550

×

CreateTwoHistogramsWithFiveBinsBFCL/parallel_129: two create_histogram calls — one per data set, 5 bins each (data order preserved).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 817 OUT758

×

EstimateElephantPopulationThreeScenariosBFCL/parallel_125: three elephant_population_estimate calls — current 500 at 2%, 1.5%, 2.5% over 10y.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN13 694 OUT945

×

FindItalianRestaurantsInNyAndLaBFCL/parallel_130: two find_restaurants calls — New York and Los Angeles, Italian, 4 results each, dietary_requirements [vegan, gluten-free]. Distinct from restaurant_find_nearby (this tool carries food_type + number, which the prompt supplies, so it is the correct choice here).1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2

Failed

IN29 150 OUT1 250

×

FindRestaurantsByDietInThreeCitiesBFCL/parallel_127: three restaurant_find_nearby calls — New York, Los Angeles, Chicago, each with dietary_preference [Vegan, Gluten-free, Dairy-free]. List order follows the prompt (matches ground truth).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN13 511 OUT854

✓

PredictEvolutionDarwinAndLamarckBFCL/parallel_126: two prediction_evolution calls — African Elephant over 5000y, Darwin then Lamarck. Tightened per methodology: the prompt explicitly says "using the Darwin model", so model == "Darwin" is required for the first call (ground truth also accepts omission via the 'Darwin' default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 385 OUT504

✓

ProjectPopulationGrowthJapanAndIndiaBFCL/parallel_124: four population_projections calls — Japan/10y and India/20y, each at the current (default) growth rate and at an explicit rate. "current growth rate" maps to the default (growth_rate omitted, accepts null or 0.01); the explicit 1.5%/2.1% rates are required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 710 OUT1 098

×

RouteThreeLegsAvoidingTollsBFCL/parallel_131: three map_routing_fastest_route calls — SF->Palo Alto->San Jose->home, all avoiding tolls. Tightened: the prompt explicitly says "avoiding toll roads", so avoid_tolls == true is required for every leg (non-default value stated in the prompt).1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN18 843 OUT881

×

ShortestCityDistanceBusWithAndWithoutTransferBFCL/parallel_136: two city_distance_find_shortest calls — NY->LA by bus, first without transfer then allowing transfer. Tightened: the prompt explicitly says "using a bus", so transportation == "bus" is required for both (also the default). allow_transfer: false/omitted for the first, true for the second.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 010 OUT835

Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 2) 14 failed

0.0%

IN296 548 OUT14 177

×

CalculateAreaOfThreeTrianglesBFCL/parallel_143: three calc_area_triangle calls — (10,5), (15,7), (20,10).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN23 797 OUT516

×

CalculateBinomialProbabilityBasketballBFCL/parallel_161: three calculate_binomial_probability calls — basketball shooter at 60%: (10,7), (15,10), (20,15). probability_of_success == 0.6 is required (prompt states 60%). ISOLATED here (away from prob_dist.binomial in ToolCalling-Parallel-D3): both tools compute binomial probability, and this prompt does not name a function, so having both visible would make the choice genuinely ambiguous. Kept apart so exactly one correct tool is available.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN14 210 OUT1 403

×

CalculateBmiInMetersForThreePeopleBFCL/parallel_138: three calculate_BMI calls (weight kg, height meters) — John, Sarah, Mike.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN19 667 OUT844

×

CalculateCelestialDistancesInMilesBFCL/parallel_149: two calculate_distance calls between celestial bodies (Mars-Venus, Mars-Jupiter), in miles. Uses the body-name distance tool (body1/body2). Tightened: prompt asks "in miles", so unit == "miles" is required (tool default is kilometers, so omission would be wrong).1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 2

Failed

IN35 751 OUT1 519

×

CalculateClockAnglesAtThreeTimesBFCL/parallel_145: three calculate_clock_angle calls — 3:15, 8:20, 11:50. Tightened: the prompt asks for results "rounded to 2 decimal places", so round_to == 2 is required (also the default).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN19 555 OUT1 315

×

CalculateFactorialOfFourNumbersBFCL/parallel_144: four math_factorial calls — 5, 3, 4, 2.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN14 180 OUT506

×

CalculateLightTravelTimeThreeLegsBFCL/parallel_147: three light_travel_time calls — 4.22, 6.1, 5.88 light years. Tightened: the prompt explicitly states the speed of light (299792458 m/s), so speed_of_light == 299792458 is required (also default).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN14 174 OUT1 064

×

CalculateSpeedForCarBicycleTrainBFCL/parallel_148: three calculate_speed calls — car 500m/25s in km/h, bicycle 1000m/200s in m/s, train 10000m/600s in km/h. Each output unit is explicitly named in the prompt, so to_unit is required.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN19 809 OUT912

×

FetchAllDataFieldsForTwoEmployeesBFCL/parallel_139: two employee_fetch_data calls — Tech Solutions, employees 12345 and 67890, each fetching all four data fields (order follows the prompt / ground truth).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN24 674 OUT857

×

FindDiCaprioDramaAndComedyMoviesBFCL/parallel_140: two imdb_find_movies_by_actor calls — Leonardo DiCaprio, Drama in 2010 and Comedy in 2012. Tightened: the prompt explicitly names each category, so category is required for both.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN18 681 OUT768

×

GetImaxAndTwoDTheaterReleasesBFCL/parallel_141: two get_theater_movie_releases calls — New York IMAX over 7 days, Los Angeles 2D over 14 days. Tightened: the prompt explicitly names each format, so format is required for both.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN18 695 OUT1 251

×

PlotTwoSineWavesBFCL/parallel_146: two plot_sine_wave calls with all five parameters supplied by the prompt.1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2

Failed

IN30 541 OUT953

×

SortFourListsAscendingAndDescendingBFCL/parallel_137: eight array_sort calls — four lists, each sorted ascending AND descending (list element order preserved as given).1 AI query 16 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 8

Failed

IN16 600 OUT1 479

×

UpdateTwoCustomersNameAndEmailBFCL/parallel_142: two update_user_info calls — users 12345 and 67890, both set name "John" and email "example@.com". update_info is a nested object. Tightened: the prompt names the 'CustomerInfo' database, so database == "CustomerInfo" is required (also the default).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN26 214 OUT790

Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 3) 1 passed10 failed

9.1%

IN165 976 OUT8 706

×

AreaUnderCurveForTwoPolynomialsBFCL/parallel_150: two mathematics_calculate_area_under_curve calls — polynomial [3,-2,1] over [-1,2] and [1,0,-1] over [0,3] (coefficient/limit order preserved).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 149 OUT724

×

BinomialProbabilitiesThreeGamesBFCL/parallel_160: three prob_dist_binomial calls — (20,5), (50,15), (100,30), each with p == 0.3 (prompt states a 0.3 win probability). The prompt explicitly names 'prob_dist.binomial'; the colliding calculate_binomial_probability tool is kept in a separate script (ToolCalling-Parallel-D2).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN12 635 OUT1 027

×

CalculateBmiSystemFourPeopleBFCL/parallel_154: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt says "use the metric system for all calculations", so system == "metric" is required (also the default).1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4

Failed

IN18 567 OUT1 142

✓

DiceRollProbabilitiesThreeScenariosBFCL/parallel_159: three probability_dice_roll calls — roll 4 three times and roll 2 twice on a six-sided die (die_sides default 6, accepts omission), then roll 7 twice on an eight-sided die (die_sides == 8 required, non-default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 962 OUT696

×

GenerateNormalVariatesTwoConfigsBFCL/parallel_158: four random_normalvariate calls — two at (mu 5, sigma 2) and two at (mu 10, sigma 3). Duplicate calls: each configuration must appear exactly twice (multiplicity 2).1 AI query 20 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 20 expectedCount = 4

Failed

IN30 788 OUT1 411

×

PredictPricesForFourHousesBFCL/parallel_157: four predict_house_price calls — New York (3/2/1500), Los Angeles (4/3/2000), Chicago (2/1/1200), Miami (3/2/1800).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN13 295 OUT963

×

RaiseTwoNumbersToPowersBFCL/parallel_152: two math_power calls — 3^5 and 2^3, no modulus (mod omitted / null).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN11 882 OUT356

×

RunLinearRegressionWithAndWithoutStandardizationBFCL/parallel_155: two run_linear_regression calls — same predictors [Age, Income, Education] and target "Spending Score", first without standardization then with. standardize is explicitly requested (false then true), so it is required for both.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 241 OUT375

×

TrainRandomForestFourConfigsBFCL/parallel_156: four random_forest_train calls on "data_random_forest" — (100,10), (200,20), (300,30), (400,40). The prompt explicitly names this function, disambiguating it from train_random_forest_classifier visible in this script.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN13 349 OUT957

×

TrainRfClassifierTwoConfigsBFCL/parallel_153: two train_random_forest_classifier calls — dataset1 (depth 10, 100 trees) and dataset2 (depth 20, 200 trees). The prompt explicitly names this function, disambiguating it from the other random-forest training tool (random_forest_train) visible in this script.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 215 OUT396

×

TriangleAreasInDifferentUnitsBFCL/parallel_151: three geometry_area_triangle calls in different units — (15,20) square meters, (25,30) square feet, (35,40) square inches. First unit is the default (accepts omission / square meters); feet and inches are non-default and required.1 AI query 7 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 3

Failed

IN20 893 OUT659

Tool Calling - Parallel Calls: Finance, Stats & Legal 3 passed10 failed

23.1%

IN234 528 OUT11 900

✓

BuildTwoDecisionTreeClassifiersBFCL/parallel_12: two model_DecisionTreeClassifier calls — gini/depth 5 and entropy/depth 10, both random_state 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 444 OUT245

×

CalculateFutureValueForThreeTermsBFCL/parallel_18: three financials_calculate_future_value calls — years 1, 5, 10 (same 1000 @ 0.05).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 3

Failed

IN26 543 OUT726

×

CalculatePresentValueForThreeTermsBFCL/parallel_14: three calculate_present_value calls — years 10, 20, 30 (same payment 1000, rate 0.05).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN18 955 OUT824

×

CapitalGainsTaxForTwoStatesBFCL/parallel_15: two calculate_capital_gains_tax calls — CA (15000/25000) and FL (20000/50000).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN18 161 OUT1 670

✓

ComputePythagorasForTwoTrianglesBFCL/parallel_10: two math_pythagoras calls — sides (3,4) and (5,12).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 387 OUT260

×

ConfidenceIntervalForTwoSampleSizesBFCL/parallel_13: two confidence_interval_calculate calls — sample sizes 50 and 150. Tightened per methodology: the prompt explicitly requests "95% confidence interval", so confidence_level == 0.95 is required (schema default is 0.9, but the prompt states the value).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN18 203 OUT1 042

✓

CourtCaseStatusAndTrialDatesBFCL/parallel_22: four court_info_get_case_status calls — cases XY1234 (LA) and GH5678 (Orange County), each queried for status (details may be omitted via the 'status' default) and for trial_date.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 758 OUT1 136

×

GetStockPriceAndVolumeForThreeCompaniesBFCL/parallel_17: three get_stock_data calls — AAPL, GOOG/GOOGL, MSFT; each with data_points {price, volume} (unordered).1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3

Failed

IN24 471 OUT925

×

LoanEligibilityForTwoInstitutionsBFCL/parallel_20: two loan_eligibility_check calls — HSBC (500k/100k) and Wells Fargo (700k/120k).1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 2

Failed

IN35 254 OUT1 138

×

MortgagePaymentForThreeLoanTermsBFCL/parallel_19: three calculate_mortgage_payment calls — loan terms 15, 20, 30 (400000 @ 0.04).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN13 601 OUT1 651

×

PredictHousePriceInTwoCitiesBFCL/parallel_11: two ml_predict_house_price calls — New York 3000 sq ft, Los Angeles 4000 sq ft.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN17 853 OUT1 082

×

ReturnOnInvestmentGainAndLossBFCL/parallel_16: two calculate_return_on_investment calls — gain 500 on 2000, loss -1000 on 5000.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN17 959 OUT535

×

SearchMoneyLaunderingConvictionsBFCL/parallel_21: two law_crimes_search calls — money laundering in San Francisco 2019 and Texas 2018.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN17 939 OUT666

Tool Calling - Parallel Calls: Legal, History & Environment 3 passed10 failed1 skipped

23.1%

IN259 771 OUT12 261

✓

BookTwoConnectingFlightsBFCL/parallel_30: two book_flight calls — San Francisco→Tokyo (2022-05-03) and Tokyo→Sydney (2022-05-18).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 061 OUT235

×

CalculateAlimonyForTwoDurationsBFCL/parallel_23: two alimony_calculator_ca_calculate calls — durations 10 and 20 years (10000/3000 income).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN14 102 OUT919

✓

CalculateWasteForFamilyAndBachelorBFCL/parallel_29: two waste_calculation_calculate calls — LA family (2 adults, 2 children) and New York bachelor (1 single). The population argument is a nested object.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 056 OUT538

×

FetchTwoHistoricalFactsBFCL/parallel_31: two history_fact_fetch calls — Treaty of Paris and Magna Carta. Optional depth/year omitted (ground truth accepts the schema defaults).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 859 OUT622

×

FindClassicalConcertsWithParkingBFCL/parallel_44: two concert_find_nearby calls — classical concerts near Berlin and Paris next Friday. Tightened per methodology: the prompt explicitly asks for concerts "with available parking", so amenities == ['Parking'] is required (this also matches the schema default, but the prompt states it). Isolated from concert_finder (parallel_43) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 16 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 2

Failed

IN50 734 OUT1 495

–

FindCourtCasesByNumberAndTypeBFCL/parallel_26: four court_case_find calls — the cross product of case numbers {67813, 71249} and types {Civil, Criminal}; case_number is a single-element array; Civil may be omitted via its default. FLAGGED as borderline (see report): the prompt "case numbers '67813','71249' for type 'Civil' and 'Criminal'" is ambiguous about the pairing; BFCL ground truth expects all four combinations, kept faithful.0 AI queries 0 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Skipped: Ambiguous call decomposition (item flaw, not model behavior): 'case numbers 67813, 71249 for type Civil and Criminal' has no single deterministic reading - two calls (both numbers per type, case_number as a list) is as valid as the BFCL 2x2 = four single-number calls; a competent human would not answer deterministically. Kept for traceability.

Skipped

IN0 OUT0

×

FindNatureReservesNearTwoCitiesBFCL/parallel_27: two nature_reserve_find_nearby calls — Berkeley (10km, picnic tables + restrooms) and Tokyo (5km, playgrounds + biking trails). Amenity list order is unordered.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN24 893 OUT1 532

×

GetCaseDetailsWithHistoryAndLitigantsBFCL/parallel_24: two law_case_get_details calls — cases 28473 and 64725. Tightened per methodology: the prompt explicitly asks for "their history and details of litigants", so include_history == true and include_litigants == true are required (schema defaults are false).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN24 740 OUT747

×

GetEventsForTwoPresidenciesBFCL/parallel_32: two us_history_events_by_presidency calls — Abraham Lincoln and George Washington. Optional start_year/end_year omitted (ground truth accepts the schema defaults).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 821 OUT688

×

GetPresidentsAndVicePresidentsByYearBFCL/parallel_33: four get_president_and_vp calls — president for 1980 and 2016, vice president for 1975 and 2011.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN14 801 OUT938

×

GetReligionInfoForTwoEmpiresBFCL/parallel_35: two ancient_empires_get_religion_info calls — Persian Empire and Mauryan Empire. Tightened per methodology: the prompt explicitly asks for "religious history and influences", so include_influences == true is required (schema default is false).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 907 OUT1 254

×

GetTemperatureForTwoCitiesBFCL/parallel_28: two get_current_and_future_temperature calls — Seattle and Los Angeles, next 3 hours.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 835 OUT257

×

LookupLawsuitsForTwoYearsBFCL/parallel_25: two lawsuit_lookup calls — Dara Inc for years 2019 and 2018.1 AI query 14 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 14 expectedCount = 2

Failed

IN47 904 OUT1 397

✓

TrackChristianityInTwoRegionsBFCL/parallel_34: two religion_history_track calls — Christianity in Egypt and Turkey, 100–1500 A.D.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 058 OUT1 639

Tool Calling - Parallel Calls: Markets & Environment 4 passed9 failed

30.8%

IN209 891 OUT10 431

×

CalculateCagrForTwoInvestmentsBFCL/parallel_176: two calculate_cagr calls — (5000,7000,5) and (8000,12000,3).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN27 783 OUT1 051

✓

CalculateEmissionSavingsForSolarAndWindBFCL/parallel_186: two calculate_emission_savings calls — solar(12 months) and wind(8 months), Midwest region. Prompt states the Midwest region explicitly -> region required (accept "Midwest"/"Midwest region").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 562 OUT375

×

CompareFutureValueWithAndWithoutContributionsBFCL/parallel_182: two finance_calculate_future_value calls — (5000,0.07,10, no contribution) and (3000,0.06,10, contribution 200). First scenario states "no additional contributions" -> null or 0.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 523 OUT1 172

×

ComputeEuclideanNormForThreePointsBFCL/parallel_183: three math_hypot calls — (5,7), (10,15), (20,25). z omitted or 0 (2D points).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 3

Failed

IN13 168 OUT519

×

EstimatePopulationsForTwoSpeciesAcrossTwoYearsBFCL/parallel_185: four estimate_population calls — Bengal Tigers/India and African Elephants/Kenya for 2021 and for 2023. NOTE: upstream ground truth's third entry lists year "" (omitted -> default 2020), which is an upstream error: the prompt explicitly asks for 2023 for "both these species". We tighten both 2023 estimates to year == 2023 (fair deterministic answer).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN14 210 OUT1 251

×

FindRootsForTwoQuadraticEquationsBFCL/parallel_184: two algebra_quadratic_roots calls — (3,7,2) and (5,-4,1).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 406 OUT857

×

GetClosingPricesForTwoCompaniesOnTwoDatesBFCL/parallel_178: four get_stock_price (by-date variant) calls — the 2x2 cross product of {Microsoft, Apple} x {2022-01-01, 2022-02-01} on NASDAQ. Upstream ground truth's first entry loosely lists company_name as [Microsoft, Apple]; the deterministic human answer is the four distinct (company, date) combinations, which we require. exchange NASDAQ (default) may be omitted.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN14 231 OUT975

×

GetDetailedAirQualityForThreeDaysBFCL/parallel_187: three get_air_quality calls — New York City, detail true, for 2023-05-05, 05-04, 05-03. Prompt asks for PM2.5/PM10/ozone/sources -> detail == true; "today/yesterday/day before" with today = May 5, 2023.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN19 271 OUT760

✓

GetFourPriceTypesForTwoCompaniesBFCL/parallel_180: eight stock_price calls — {Microsoft, Apple} x {Open, Close, High, Low}, days 30. Prompt explicitly lists all four data types, so data_type must be set explicitly (required) to distinguish the eight calls; omission (default 'Close') cannot represent the four requested types.1 AI query 8 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Passed

IN9 442 OUT659

×

GetPricePerOunceForFourMetalsBFCL/parallel_177: four get_metal_price calls — gold, silver, platinum, palladium, measure 'ounce'. Prompt asks "price per ounce" -> measure == "ounce" required.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN13 970 OUT682

✓

GetStockPricesForFourCompaniesAndDurationsBFCL/parallel_181: four get_stock_prices calls — Apple(1 week), Microsoft(2 weeks), Amazon(3 weeks), Tesla(1 month). Each 'companies' argument is a single-element list.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 838 OUT257

✓

GetTrafficInfoForThreeLegsWithModesBFCL/parallel_188: three get_traffic_info calls — NY->LA (driving), LA->SF (bicycling), SF->NY (transit). First leg explicitly says "driving" -> mode == "driving" required (tightened; also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 763 OUT359

×

ProjectFutureValueForThreeStocksBFCL/parallel_175: three portfolio_future_value calls — AAPL(5000,0.07,5), MSFT(8000,0.06,7), AMZN(10000,0.08,10).1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 3

Failed

IN44 724 OUT1 514

Tool Calling - Parallel Calls: Mathematics & Physics (set 1) 5 passed8 failed

38.5%

IN162 627 OUT8 020

×

AreaOfTwoRightTrianglesInMetersBFCL/parallel_68: two calculate_area calls — right triangles base/height 12x15 and 18x24 in meters. unit required (tool default is 'cm'; the prompt states meters/square meters).1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 2

Failed

IN16 751 OUT578

✓

CheckPlayerStatusAcrossThreeSeasonsBFCL/parallel_49: three player_status_check calls — player 3142 in RocketLeague for seasons 2017/2018/2019. Seasons are explicitly given, so each is required (default 2023 not applicable).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 075 OUT448

✓

CircumferenceOfFourCirclesInCmBFCL/parallel_65: four calculate_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt gives every radius in cm (e.g. "5cm") and the tool default unit is 'm', so a correct call must pass unit == 'cm'/'centimeter' (ground truth requires it on the first call and merely allows omission on the rest; we require it consistently since it is stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 217 OUT563

×

ConvertEuroAndPoundsToDollarsBFCL/parallel_59: two get_conversion_cost calls — 15000 Euro->dollars and 200 pounds->dollars.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 245 OUT474

×

FinalVelocityForFourDropsBFCL/parallel_81: four calculate_final_velocity calls — heights 10/20/15/25, initial_velocity 0 (stated), gravity 9.81 (default, may be omitted).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN13 445 OUT1 055

×

FindItalianAndJapaneseRestaurantsBFCL/parallel_52: two restaurant_finder calls — Italian in New York (preferences ['Vegetarian']) and Japanese in Los Angeles (delivery; ['Delivery'] or omitted since default is ['Delivery']).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN16 857 OUT566

×

FindRootsOfTwoQuadraticsAlgebraBFCL/parallel_62: two algebra_quadratic_roots calls — 3x^2+4x+2 and 5x^2-7x+3.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 461 OUT826

×

FindSoupAndVeganSaladRecipesBFCL/parallel_51: two recipe_search_find calls — Chicken Noodle Soup (diet omitted/default 'Keto') and a Vegan Salad (diet == 'Vegan', dish name accepts several phrasings).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN16 697 OUT554

×

GcdOfJohnAndMaryNumbersBFCL/parallel_80: two math_gcd calls — gcd(36,48) and gcd(60,96). Same tool as parallel_76.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN21 825 OUT696

✓

GcdOfTwoPairsBFCL/parallel_76: two math_gcd calls — gcd(45,60) and gcd(81,27).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 882 OUT146

×

GetPokemonMovesAndCheckSolarBeamBFCL/parallel_48: two PokemonGO_get_moves calls — Pikachu's move list (move omitted/default 'Run') and a check whether Bulbasaur can learn 'Solar Beam'.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 158 OUT890

✓

PrimeFactorsFormattedAndArrayBFCL/parallel_74: four get_prime_factors calls — numbers 4567 and 7890, each formatted true and false (user wants "a formatted string as well as an array").1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 164 OUT1 087

✓

SaveGameProgressEasyAndHardBFCL/parallel_50: two game_save_progress calls — stage 7 easy and stage 3 hard; level omitted/default 'user'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 850 OUT137

Tool Calling - Parallel Calls: Mathematics & Physics (set 2) 3 passed10 failed

23.1%

IN225 047 OUT12 201

×

AreaOfThreeCirclesInMetersBFCL/parallel_66: three geometry_area_circle calls — radii 5/10/15 in meters (default is meters, so units may be omitted).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN18 243 OUT786

✓

BookTwoHotelRoomsBFCL/parallel_57: two hotel_booking_book calls — Sheraton (NY, 2 adults/1 child) and Marriott (LA, 1 adult/2 children).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 387 OUT352

×

CheckAvailabilityForTwoHotelsBFCL/parallel_56: two hotel_booking_check_availability calls — Queens Hotel (Berlin) and Royal Hotel (Paris), each with check-in/out dates (several date formats accepted).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN23 370 OUT898

×

CircumferenceOfFourCirclesInMetersBFCL/parallel_70: four geometry_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt asks for output in meters and gives radii in m, but the tool default unit is 'cm', so a correct call must pass units == 'm'/'meters' (ground truth requires it on the first call and only allows omission on the rest; we require it consistently since it is explicitly stated).1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 4

Failed

IN13 140 OUT653

×

DerivativeOfPolynomialAndItsDerivativeBFCL/parallel_71: two calculate_derivative calls — 3x^3-2x^2+5x-7 at x=4, then its derivative 9x^2-4x+5 at x=2.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN21 609 OUT1 017

×

FinalVelocityForThreeDragRaceRoundsBFCL/parallel_83: three final_velocity calls — (v0=0,a=5,t=10), (v0=10,a=7,t=8), (v0=20,a=4,t=12).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN18 689 OUT892

×

FindGroceryStoresByCriteriaBFCL/parallel_55: two grocery_store_find_by_criteria calls — NYC ['24 hours'] and San Diego ['Home Delivery'].1 AI query 14 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 14 expectedCount = 2

Failed

IN39 409 OUT1 672

×

GetTwoCookingRecipesBFCL/parallel_53: two get_cooking_recipe calls — Lasagne Bolognese for 4 and Caesar Salad for 2.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN22 435 OUT1 830

×

GetTwoExchangeRatesBFCL/parallel_58: two get_exchange_rate calls — USD->AUD and USD->CAD.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN17 269 OUT485

×

HighestCommonFactorOfFourPairsBFCL/parallel_77: four math_hcf calls — (45,60), (90,120), (36,48), (72,96). FLAGGED (see final report): the prompt is phrased as chaining ("use that result to find the HCF with...") but the ground truth treats it as four INDEPENDENT parallel calls on the literal pairs; a human might interpret it as sequential composition. Kept BFCL-faithful (four independent literal-pair calls) since parallel semantics preclude feeding one call's result into another.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 4

Failed

IN13 332 OUT1 642

✓

OrderSameItemsFromTwoWholeFoodsLocationsBFCL/parallel_54: two whole_foods_order calls — same two items (order-independent) at downtown and uptown, size large.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 200 OUT229

✓

PrimeFactorizeAsDictionaryBFCL/parallel_79: two prime_factorize calls — 4567 and 7890, return_type 'dictionary' (explicitly requested).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 176 OUT880

×

SolveTwoQuadraticEquationsBFCL/parallel_63: two solve_quadratic_equation calls — (5,6,1) and (3,2,1). Prompt names this function explicitly.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 788 OUT865

Tool Calling - Parallel Calls: Mathematics & Physics (set 3) 11 failed1 broken

0.0%

IN608 969 OUT19 402

×

AreaOfTwoCirclesInMetersBFCL/parallel_67: two geometry_calculate_area_circle calls — radii 5 and 10 in meters. unit required (tool default is 'cm'; the prompt states meters).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN17 059 OUT628

!

AreaOfTwoTrianglesBFCL/parallel_69: two calculate_triangle_area calls — base/height 10x5 and 8x6.1 AI query 55 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Broken

IN308 481 OUT6 258

×

DerivativeOfTwoFunctionsAtValuesBFCL/parallel_73: two calculus_derivative calls — 3x^2+2x-1 at 5 (variable x, default) and 4y^3-3y^2+2y-1 at 3 (variable y).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 608 OUT537

×

DisplacementForThreeCarsBFCL/parallel_84: three calculate_displacement calls — (15,7,accel 3.5), (20,10,accel 2.0), (25,8,no acceleration). The third car travels "without any acceleration" so acceleration == 0 (also the default; omission accepted).1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3

Failed

IN24 091 OUT1 281

×

EuclideanNormForThreePointsBFCL/parallel_61: three math_hypot calls — (3,4) and (6,8) with z omitted/0, and (9,12,15).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN18 101 OUT869

×

FactorialOfThreeNumbersBFCL/parallel_60: three math_factorial calls — 5, 7, 9.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN12 647 OUT506

×

FinalSpeedForTwoFreeFallObjectsBFCL/parallel_85: two calculate_final_speed calls — initial_speed 0 for 10 s and initial_speed 5 for 7 s; gravity -9.81 (default, may be omitted).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 611 OUT649

×

GcdViaNumberTheoryBFCL/parallel_78: two number_theory_gcd calls — (45,60) and (81,63). Prompt names this function explicitly.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN22 045 OUT750

×

IntegrateTwoFunctionsWithTwoMethodsBFCL/parallel_72: four integrate calls — x**3 on [2,5] and 2x**2+3x-1 on [-1,3], each with trapezoid (default, may be omitted) and simpson (explicit).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN13 622 OUT1 616

×

PrimeFactorsOfThreeNumbersBFCL/parallel_75: three number_analysis_prime_factors calls — 45, 100, 150.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3

Failed

IN40 643 OUT1 168

×

SolveFourQuadraticsWithRootTypeBFCL/parallel_64: four solve_quadratic calls — (2,5,3) all roots, (1,-3,2) real, (4,-7,3) all roots, (1,2,1) real. 'all' is the default so it may be omitted; 'real' is explicitly requested and required.1 AI query 40 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 40 expectedCount = 4

Failed

IN93 705 OUT4 199

×

VelocityForTwoCyclingDaysBFCL/parallel_82: two calculate_velocity calls — 120 km in 5 h and 150 km in 6 h; unit km/h (default, may be omitted).1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 2

Failed

IN33 356 OUT941

Tool Calling - Parallel Calls: Mixed Domains 3 passed7 failed

30.0%

IN123 762 OUT6 687

×

CalculateBmiForTwoPeopleBFCL/parallel_4: two calculate_bmi calls — 6 ft / 80 kg and 5.6 ft / 60 kg.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN11 276 OUT305

×

CalculateEmForceForTwoTimeIntervalsBFCL/parallel_1: two calculate_em_force calls — same field/area, d_time 4 then 10.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN11 330 OUT767

✓

CalculateFactorialForThreeNumbersBFCL/parallel_7: three math_factorial calls — 5, 10, 15.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 342 OUT192

×

CalculateResistanceForCopperAndAluminumBFCL/parallel_2: two calculate_resistance calls — copper then aluminum, same length/area. Tightened per methodology: the prompt explicitly names "resistivity of copper", so the first call must pass resistivity == "copper" (ground truth also accepts omission via the 'copper' default, but an explicitly-stated value must be supplied). Aluminum is non-default → required.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN11 324 OUT749

×

CalculateSalesTaxForThreeCitiesBFCL/parallel_6: three calculate_sales_tax calls — Chicago/IL $30.45, Sacramento/CA $52.33, Portland/OR $11.23.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN16 645 OUT1 076

×

FetchPopulationForCitiesStateAndCountryBFCL/parallel_8: four database_us_census_get_population calls — NYC (city), LA (city), Alaska (state), USA (country); year defaults to 2000 (may be omitted). Requires area-type inference.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN12 165 OUT732

×

FindMovieShowingsForTwoMoviesBFCL/parallel_9: two find_movie_showing calls near San Diego — Tenet at 5 pm and No Time To Die at 7:30 pm.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 2

Failed

IN23 218 OUT663

✓

GetSequenceAnd3DForThreeProteinsBFCL/parallel_3: three protein_info_get_sequence_and_3D calls — HbA1c, normal + rat hemoglobin. Tightened per methodology: the prompt explicitly asks for "their 3D models", so model_3d == true is required (ground truth also accepts omission via the default of true).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 421 OUT1 385

×

ListShowRatingsOnNetflixAndHuluBFCL/parallel_5: two streaming_services calls — Netflix['Friends'] and Hulu['The Office','Stranger Things'] (Hulu show order unordered), both sorted by rating. Must group shows per service.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN15 775 OUT672

✓

PlayTwoArtistsForGivenDurationsBFCL/parallel_0: two spotify_play calls — Taylor Swift for 20 min and Maroon 5 for 15 min.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 266 OUT146

Tool Calling - Parallel Calls: Physical & Life Sciences (set 1) 2 passed10 failed

16.7%

IN194 826 OUT12 149

×

ElectricFieldForTwoChargesBFCL/parallel_94: two calculate_electric_field calls — charge 5/distance 2 and charge 3/distance 4. "Conducted in a vacuum" = the default permitivity → may be omitted or 0.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 2

Failed

IN18 889 OUT355

×

ElectromagneticForceInTwoMediaBFCL/parallel_96: two electromagnetic_force calls — same charges 5,10 and distance 2, differing by medium_permittivity. Vacuum call (8.854e-12) equals the default → may be omitted or 8.854e-12; the second medium (5e-12) is non-default → required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 289 OUT1 368

×

EstimateTravelTimeForThreeRoutesBFCL/parallel_93: three route_estimate_time calls — (1) NYC->Miami via Philadelphia/Washington D.C./Atlanta, (2) NYC->Miami skipping Atlanta (via Philadelphia/Washington D.C.), (3) Philadelphia-> Miami via Washington D.C. Stop orderings from ground truth accepted. FLAG parallel_93: BORDERLINE. Ground truth allows many stop-list variants, some including the start city itself as a stop; a competent human's stop set is not fully deterministic. Kept BFCL-faithful by accepting the natural stop orderings (start city excluded from stops).1 AI query 7 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 3

Failed

IN22 167 OUT1 152

✓

FinalSpeedForTwoDroppedObjectsBFCL/parallel_88: two calculate_final_speed calls — v0 0/height 10 and v0 5/height 20. gravity stated as 9.8 (the default) so it may be omitted or 9.8 per ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 420 OUT395

×

FinalVelocityForTwoCarsBFCL/parallel_87: two calculate_final_velocity calls — Car A (v0 0, a 6, t 10), Car B (v0 20, a 4, t 15).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN23 155 OUT772

×

FinalVelocityFromDistanceForTwoObjectsBFCL/parallel_86: two kinematics_final_velocity_from_distance calls — accel 5/dist 100 and accel 10/dist 200. "Both objects start from rest" → initial_velocity may be omitted or 0 (default).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 980 OUT595

×

FindVeganRestaurantsInThreeCitiesBFCL/parallel_91: three vegan_restaurant_find_nearby calls — LA, SF, Seattle, each with operating_hours 22 (prompt: "open until at least 22:00", a non-default value → required).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN18 581 OUT1 030

✓

GenerateFourTravelItinerariesBFCL/parallel_90: four travel_itinerary_generator calls — Tokyo/7/200/urban, Paris/10/150/history, Sydney/5/100/nature, Rome/12/180/culture. Tokyo's "urban" equals the default → omit or "urban".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 791 OUT2 073

×

GetDirectionsForRoundTripBFCL/parallel_89: three get_directions calls — SF->Palo Alto (fastest), Palo Alto->Golden Gate Bridge (scenic), Golden Gate Bridge->SF (fastest). route_type explicitly stated each time.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN18 833 OUT1 042

×

MagneticFieldForTwoLoopsBFCL/parallel_95: two calculate_magnetic_field calls — current 10/radius 0.5 and current 15/radius 1.0. "Same as in free space" = default permeability → may be omitted or 0.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN23 230 OUT1 085

×

ResonantFrequencyForTwoCircuitsBFCL/parallel_97: two calculate_resonant_frequency calls — L 0.005/C 1e-07/round 3 and L 0.007/C 2e-07/round 4. Rounding is stated explicitly → required.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 157 OUT1 424

×

ShortestDrivingDistanceRoundTripBFCL/parallel_92: three get_shortest_driving_distance calls — NYC->LA, LA->Miami, Miami->NYC, all in miles (prompt explicitly "in miles" → non-default unit required).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN13 334 OUT858

Tool Calling - Parallel Calls: Physical & Life Sciences (set 2) 7 passed6 failed

53.8%

IN178 472 OUT13 228

✓

AbsolutePressureAtTwoAltitudesBFCL/parallel_102: two calc_absolute_pressure calls — gauge 2.5 with atm 1 (default; may be omitted) and gauge 2.5 with atm 0.85 (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 525 OUT2 954

×

BoilingMeltingPointsAcrossSeaLevelsBFCL/parallel_100: four get_boiling_melting_points calls — water@0, iron@1000, water@1000, iron@0. FLAG parallel_100: BORDERLINE. The prompt ("water and iron at sea levels of 0 and 1000 meters respectively") reads most naturally as TWO calls (water@0, iron@1000), but the BFCL ground truth is the full 2x2 cross product (FOUR calls). Kept BFCL-faithful to the ground truth; a competent human could reasonably answer with two calls, so this item's expected call count is not fully deterministic.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 986 OUT715

✓

CellInfoDetailedAndBriefBFCL/parallel_108: two biology_get_cell_info calls — neuron with detailed=true (prompt: "detailed description"), muscle with detailed=false ("less detailed", the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 412 OUT868

✓

DensityForTwoSubstancesBFCL/parallel_101: two calculate_density calls (mass/volume schema) — 10 kg / 2 m³ and 15 kg / 3 m³. unit "kg/m³" equals the default → may be omitted or "kg/m³".1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 540 OUT237

✓

ElectricFieldStrengthAtSeveralDistancesBFCL/parallel_98: four calculate_electric_field_strength calls — charge 2, distances 0.5/1/2 in vacuum (default medium), plus distance 1 in air. The distance-1 vacuum and distance-1 air calls are distinguished by medium. Vacuum equals the default → may be omitted or "vacuum"; "air" is required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 835 OUT1 266

×

EntropyChangeForTwoConditionsBFCL/parallel_103: two entropy_change_calculate calls — substance A, mass 2, temps 25->75 and 10->50. pressure 1 equals the default → may be omitted or 1.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 2

Failed

IN27 722 OUT1 282

×

FetchDnaSequenceThreeWaysBFCL/parallel_106: three fetch_DNA_sequence calls for id XYZ123 — default fasta/0, then genbank format, then 500 bp upstream. Defaults (fasta, upstream 0) may be omitted.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN18 831 OUT631

×

FunctionLookupForTwoMoleculesBFCL/parallel_110: two cell_biology_function_lookup calls — ATP in mitochondria and DNA in nucleus, both with specific_function=true (prompt asks for a specific function within the organelle).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN18 055 OUT1 007

✓

HeatCapacityForThreeConditionsBFCL/parallel_105: three calc_heat_capacity calls — 300K/2.5m³, 350K/2.5m³, 300K/1.5m³, all for air. "air" equals the default gas → may be omitted or "air".1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 721 OUT340

×

MolecularWeightForTwoCompoundsBFCL/parallel_111: two calculate_molecular_weight calls — Glucose (C6H12O6) and Sucrose (C12H22O11), both in grams/mole (prompt explicitly requests that unit; ground truth also accepts "g/mol").1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 097 OUT923

×

PhaseChangeEnergyForWaterBFCL/parallel_99: four thermo_calculate_energy calls — 500 g water, phase melting/freezing/ vaporization/condensation. Substance "water" equals the default → may be omitted or "water".1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 4

Failed

IN27 469 OUT1 481

✓

ProteinSequencesForTwoGenesTwoSpeciesBFCL/parallel_107: four get_protein_sequence calls — BRCA1 and BRCA2 in Homo sapiens (default species, may be omitted) and in Pan troglodytes (non-default → required).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 712 OUT744

✓

ProteinsInThreeCompartmentsBFCL/parallel_109: three cellbio_get_proteins calls — nucleus/mitochondria/cytoplasm, each with include_description=true (prompt asks for "a brief description of each protein").1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 567 OUT780

Tool Calling - Parallel Calls: Physical & Life Sciences (set 3) 5 passed7 failed1 broken

38.5%

IN165 366 OUT21 526

×

AnalyzeDnaForThreeMutationTypesBFCL/parallel_114: six analyze_dna_sequence calls — two sequences ("AGCTTAGCTA" and "AGCTTAGGCTA") against reference "AGCTTAGCTA", each checked for insertion (default, may be omitted), deletion, and substitution mutations.1 AI query 7 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 6

Failed

IN22 851 OUT1 136

×

DiabetesPredictionForFourPeopleBFCL/parallel_113: four diabetes_prediction calls for four people with distinct weight/height/activity.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN14 306 OUT1 457

×

EntropyChangeIsothermalAndNotBFCL/parallel_104: two calculate_entropy_change calls — same 300->350 K and heat_capacity 4.18, once isothermal (the default true, may be omitted) and once not isothermal (false, required).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 370 OUT1 358

×

ForestGrowthForTwoRegionsBFCL/parallel_120: two forest_growth_forecast calls — Amazon Rainforest over 10 years and Boreal Forests of Canada over 20 years, excluding human impact (false = the default, may be omitted).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 175 OUT968

!

GenerateFourDnaSequencesBFCL/parallel_123: four generate_DNA_sequence calls — length 500 with a single-nucleotide preference of A, T, C, G respectively.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Broken

IN8 826 OUT10 200

✓

GeneticSimilarityInTwoFormatsBFCL/parallel_115: four genetics_calculate_similarity calls — human vs chimpanzee and human vs gorilla, each in percentage (default, may be omitted) and fraction. species1 accepts "human"/"Human".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 814 OUT617

×

GenotypeFrequenciesForThreeGenotypesBFCL/parallel_116: three calculate_genotype_frequency calls — allele frequency 0.7, genotypes AA/Aa/aa.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN13 613 OUT711

✓

IdentifyFourBirdsBFCL/parallel_119: four identify_bird calls — small blue/forest (default size, may be omitted), large black/lake, medium brown/desert, large green/tropical rainforest.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 922 OUT1 707

✓

MutationTypeForTwoSnpsBFCL/parallel_112: two mutation_type_find calls — rs123456 in Homo sapiens (default species, may be omitted) and rs7891011 in Canis lupus familiaris (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 541 OUT593

×

PopulationDensityForChinaTwoYearsBFCL/parallel_117: two calculate_density calls (country/population schema) — China in 2000 (pop 1.267 billion) and 2010 (pop 1.341 billion), land area 9.597 million km² both times.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN22 409 OUT839

✓

PrecipitationStatsForThreeTimeFramesBFCL/parallel_118: three ecology_data_precipitation_stats calls — Amazon rainforest for six_months, year, five_years.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 614 OUT678

×

TurtlePopulationForTwoYearsBFCL/parallel_121: two ecology_get_turtle_population calls — Galapagos Islands in 2015 and 2020, both with species info requested (species=true; years stated explicitly → required).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 199 OUT721

✓

VehicleEmissionsForThreeVehiclesBFCL/parallel_122: three calculate_vehicle_emission calls — 15000 miles each: gas with default factor 1.4 (may be omitted), diesel with factor 2.7, EV with factor 0.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 726 OUT541

Tool Calling - Parallel Calls: Services & Lookups 2 passed10 failed

16.7%

IN220 494 OUT9 521

✓

CalculateShortestAndScenicDistancesForTwoLegsBFCL/parallel_190: four calculate_shortest_distance calls — {NYC->LA, LA->Miami} x {Shortest, Scenic}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 372 OUT717

×

FetchNewsForTwoTopicsInEuropeBFCL/parallel_192: two get_news calls — Climate Change and Artificial Intelligence, 5 articles each, region Europe. Region "Europe" explicitly requested -> required (accept "Europe"/"EU").1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 458 OUT637

×

FindLibrariesWithFacilitiesInThreeCitiesBFCL/parallel_191: three public_library_find_nearby calls — NY[Reading Room,Fiction], LA[Wi-Fi,Children Section], Chicago[Cafe,Reading Room]. Facilities compared as an unordered pair.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN18 251 OUT918

×

FindParksWithAmenitiesInThreeCitiesBFCL/parallel_189: three parks_find_nearby calls — NY[Tennis Court,Picnic Area], LA[Playground,Running Track], Chicago[Tennis Court,Playground]. Amenities compared as an unordered pair.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN18 201 OUT888

×

FindUpcomingEventsInThreeCitiesBFCL/parallel_194: three event_finder_find_upcoming calls — jazz/LA(14d), rock/Chicago(10d), classical/Boston(7d). All day windows explicitly stated -> days_ahead required (7 for Boston, which also equals the default).1 AI query 21 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 21 expectedCount = 3

Failed

IN43 983 OUT1 777

×

GetCurrentWeatherForFourCitiesBFCL/parallel_199: four get_current_weather calls — New York, Los Angeles, London, Tokyo, with both temperature and humidity included. Prompt explicitly asks to include both -> flags == true (also the default).1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4

Failed

IN18 961 OUT1 060

×

GetDetailedBriefsForTwoMoviesBFCL/parallel_195: two movie_details_brief calls — Inception and The Dark Knight, extra_info true. Prompt asks for Director/Cast/Awards -> extra_info == true required.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN17 067 OUT658

×

GetLawsuitDetailsForTwoCasesBFCL/parallel_196: two get_lawsuit_details calls — case 12345 (NY Supreme Court, with verdict) and case 67890 (LA Superior Court, no verdict). "would like the verdict" -> true; "do not need" -> null/false.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN22 655 OUT763

×

GetLawsuitInfoForTwoCasesByYearBFCL/parallel_197: two lawsuit_info calls — 12345ABC (2018, New York) and 67890XYZ (2019, California). Years are explicitly stated -> required. First location "New York" may be omitted (matches the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 646 OUT471

×

GetStockPricesForTwoCompaniesOverDaysBFCL/parallel_179: two get_stock_price (by-days variant) calls — Apple on NASDAQ for 10 days, Microsoft on NYSE for 15 days. Both exchanges are explicitly named -> required (NASDAQ / NYSE). The tool's 'company' field is a ticker symbol; company names are accepted as synonyms.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN17 181 OUT495

×

SearchLawsuitsForTwoEntitiesInCaliforniaBFCL/parallel_198: two lawsuit_search calls — Google/Santa Clara and Facebook/San Mateo, state California. Prompt states "both in the state of California" -> state required (accept "California"/"CA").1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN22 342 OUT833

✓

SendTwoEmailsWithCcAndBccBFCL/parallel_193: two send_email calls — Project Update to john.doe (cc manager, bcc hr) and Meeting Reminder to jane.doe (cc assistant, bcc hr). Bodies matched exactly.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 377 OUT304

Tool Calling - Parallel Calls: Statistics & Finance 3 passed10 failed

23.1%

IN305 753 OUT15 941

✓

CalculateBinomialProbabilitiesForBasketballShotsBFCL/parallel_163: three calc_binomial_prob calls — (10,6,0.6), (10,6,0.5), (15,6,0.5).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 161 OUT1 186

×

CalculateCompoundInterestForThreePeriodsBFCL/parallel_170: three calculate_compound_interest calls — principal 5000, rate 0.025, time 2/3/5. Prompt states "compounded quarterly, 4 times in a year" -> n == 4 required (tightened; also ground truth).1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3

Failed

IN43 422 OUT1 675

×

CalculateDiscountedCashFlowForTwoBondsBFCL/parallel_169: two calculate_discounted_cash_flow calls — (50,5,0.05) and (60,7,0.04). Prompt explicitly states "face value of the bond is $1000" for both -> face_value == 1000 required (ground truth also accepts omission via the default, but an explicitly-stated value must be supplied).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN19 205 OUT1 525

×

CalculateEventProbabilitiesForCardDeckBFCL/parallel_162: three probability_of_event calls — Ace 4/52 as decimal, heart 13/52 as decimal, red card 26/52 as ratio. format_as_ratio: "as a decimal" -> false or omitted; "as a ratio" -> true (required).1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 3

Failed

IN23 631 OUT785

×

CalculateQuarterlyDividendForThreeScenariosBFCL/parallel_168: three finance_calculate_quarterly_dividend_per_share calls — (5M,2M), (6M,2M), (6M,2.5M).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN20 201 OUT1 094

×

CalculateReturnOnEquityForTwoCompaniesBFCL/parallel_171: two calculate_return_on_equity calls — (1M,5M,200k) and (2M,10M, no dividends). Second company "did not pay any dividends" -> dividends_paid null or 0.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2

Failed

IN30 625 OUT1 161

×

CalculateRoiForThreeStocksBFCL/parallel_174: three calculate_return_on_investment calls — (150,180,20), (200,210,30), (250,300,40). Dividends are explicitly stated per stock -> required.1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 18 expectedCount = 3

Failed

IN39 881 OUT1 701

×

ComputeRSquaredForTwoRegressionModelsBFCL/parallel_167: two linear_regression_get_r_squared calls on the same CSV — swap education/credit_score as dependent variable. Independent-variable order follows the prompt/ground truth.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 2

Failed

IN42 989 OUT1 192

✓

PredictFutureValueForTwoInvestmentsBFCL/parallel_172: two finance_predict_future_value calls — (5000,0.05,10) and (7000,0.04,15). Prompt states "compound interest annually" -> compounding_periods_per_year == 1 required (tightened; also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 968 OUT631

×

PredictHousePricesForThreeHousesBFCL/parallel_166: three predict_house_price calls — NY 2500/3/2000, LA 3000/3/2005, Chicago 2000/2/1995. The second house is described as "similar" with only area/year/location changed, so rooms == 3 per ground truth.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN14 393 OUT945

×

PredictProfitForTwoFundsBFCL/parallel_173: two investment_predictProfit calls — (5000,0.07,5) and (8000,0.05,7).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 778 OUT1 001

×

RunChiSquaredTestForTwoContingencyTablesBFCL/parallel_164: two chi_squared_test calls — tables [45,55,35,65] and [30,70,50,50], alpha 0.05. Prompt states "significance level of 0.05 for both" -> alpha == 0.05 required (tightened). Table order is fixed.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2

Failed

IN30 349 OUT1 628

✓

RunTTestAtTwoSignificanceLevelsBFCL/parallel_165: two t_test calls — same datasets, alpha 0.05 then 0.01 (both stated -> required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 150 OUT1 417

Tool Calling - Simple: Arts, Venues & Music (set 1) 8 failed

0.0%

IN264 402 OUT6 151

×

SimpleConcertDetailsBeyonceBFCL/simple_python_286 (concert.get_details): artist=Beyonce, location=San Diego, date=April 2022.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN20 822 OUT462

×

SimpleConcertInfoPopNyBFCL/simple_python_284 (concert_info.get): location=New York, date=next month, genre=Pop.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN51 790 OUT1 109

×

SimpleGenerateMelodyC4BFCL/simple_python_291 (music_generator.generate_melody): key=C, start_note=C4, length=16, tempo=120 or omitted.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN29 170 OUT730

×

SimpleKeySignatureCsharpBFCL/simple_python_298 (music_theory.key_signature): key='C#', scale_type=major or omitted.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN32 768 OUT571

×

SimpleMuseumHoursLouvreBFCL/simple_python_273 (museum.get_hours): name='Louvre Museum', location=Paris, day omitted or Monday.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN28 712 OUT644

×

SimplePaintNeededWall30x12BFCL/simple_python_259 (calculate_paint_needed): coverage_rate=400, length=30, height=12.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN24 840 OUT576

×

SimpleReligionHistoryChristianity16cBFCL/simple_python_252 (get_religion_history): religion=Christianity, century=16, sort_by=importance, count=10.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN24 952 OUT1 074

×

SimpleSculptureInfoPlensaBFCL/simple_python_263 (get_sculpture_info): artist_name='James Plensa', detail=true.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN51 348 OUT985

Tool Calling - Simple: Arts, Venues & Music (set 2) 8 failed

0.0%

IN685 525 OUT11 363

×

SimpleComposeMelodyCFGBFCL/simple_python_292 (compose_melody): progression=[C,F,G], measures=4, instrument=Piano or omitted.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN37 669 OUT865

×

SimpleConcertFindDetailsWeekndBFCL/simple_python_290 (concert.find_details): artist=The Weeknd, month=December, year=2022 or omitted.1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN72 942 OUT1 179

×

SimpleFindConcertRockChicagoBFCL/simple_python_285 (find_concert): location=Chicago, price=100, genre=Rock.1 AI query 44 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 44

Failed

IN288 801 OUT3 096

×

SimpleMuseumInfoMetBFCL/simple_python_274 (museum_info): museum_name=Metropolitan Museum of Art, info_type=opening_hours or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 427 OUT414

×

SimpleMusicalScaleCsharpBFCL/simple_python_299 (musical_scale): key='C#'/'C sharp', scale_type=major or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 600 OUT395

×

SimplePaintRequirementWallExclusionBFCL/simple_python_260 (paint_requirement.calculate): area {width=20,height=12}, paint_coverage=350, exclusion {type=window, area=15}.1 AI query 14 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 14

Failed

IN64 821 OUT1 507

×

SimpleReligionHistoryChristianity300To400BFCL/simple_python_254 (get_religion_history): religion=Christianity, start_year=300, end_year=400, event_type=all or omitted.1 AI query 28 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 28

Failed

IN158 500 OUT3 002

×

SimpleSculptureDetailsDavidBFCL/simple_python_264 (sculpture.get_details): artist=Michelangelo, title=David, detail=size.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN16 765 OUT905

Tool Calling - Simple: Arts, Venues & Music (set 3) 1 passed8 failed

11.1%

IN388 734 OUT7 811

×

SimpleChordProgressionKeyBFCL/simple_python_297 (music.theory.chordProgression): progression=[I,V,vi,IV], bools any.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 915 OUT517

×

SimpleCompoundInterest10kBFCL/simple_python_269 (calculate_compound_interest): principle=10000, interest_rate=0.05, time=10, compounds_per_year=1 or omitted.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN26 380 OUT777

×

SimpleConcertSearchClassicalLaBFCL/simple_python_287 (concert.search): genre=classical, location=Los Angeles/LA, date='this weekend', price_range=cheap.1 AI query 29 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 29

Failed

IN168 956 OUT2 284

×

SimpleCreateMixCMajorBFCL/simple_python_293 (music_composer.create_mix): scale='C Major', note_duration=quarter, track_length=180.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 262 OUT233

✓

SimpleGenerateCircleImageRedBFCL/simple_python_256 (generate_circle_image): radius=50, color=Red, background=white or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 688 OUT83

×

SimpleMuseumWorkingHoursLouvreBFCL/simple_python_276 (museum_working_hours.get): museum=Louvre, location=Paris, day=Monday or omitted.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN30 116 OUT707

×

SimpleRetrieveReligionInfoBuddhismBFCL/simple_python_253 (retrieve_religion_info): religion_name=Buddhism, detail_level=full.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN48 740 OUT1 765

×

SimpleSculptureSearchChicago19cBFCL/simple_python_265 (sculpture_search): location=Chicago, time_frame='19th century', material=all or omitted.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN53 701 OUT912

×

SimpleTimeDifferenceSfSydneyBFCL/simple_python_250 (get_time_difference): place1=San Francisco/SF, place2=Sydney.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN29 976 OUT533

Tool Calling - Simple: Arts, Venues & Music (set 4) 9 failed

0.0%

IN480 194 OUT8 252

×

SimpleBuildingDimensionsEmpireStateBFCL/simple_python_270 (building.get_dimensions): building_name=Empire State Building, unit=feet.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 140 OUT566

×

SimpleConcertFindNearbySeattleBFCL/simple_python_289 (concert.find_nearby): location=Seattle, genre=jazz.1 AI query 8 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN32 768 OUT630

×

SimpleCreateChordProgressionCBFCL/simple_python_294 (music_generation.create_chord_progression): key=C, chords=4, progression_type=major or omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 188 OUT282

×

SimpleEarliestReferenceJesusBFCL/simple_python_251 (get_earliest_reference): name='Jesus Christ', source='historical records'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 652 OUT468

×

SimpleFindInstrumentAcousticBFCL/simple_python_280 (find_instrument): budget=1000, type=acoustic, make omitted.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN51 296 OUT971

×

SimpleGuitarPriceLesPaulBFCL/simple_python_283 (guitar_price.find): model='Gibson Les Paul', condition=Excellent, location=Chicago.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 644 OUT559

×

SimpleIdentifyColorSeaGreenBFCL/simple_python_257 (identify_color_rgb): color_name='Sea Green', standard=basic or omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN16 741 OUT289

×

SimpleMuseumInfoBritishDateBFCL/simple_python_277 (museum_info): museum='The British Museum', date=2023-06-20, information=[opening_hours,ticket_price] (either order).1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN17 558 OUT591

×

SimpleSculptureValueThinkerBFCL/simple_python_266 (get_sculpture_value): sculpture='The Thinker', artist=Rodin.1 AI query 45 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 45

Failed

IN301 207 OUT3 896

Tool Calling - Simple: Arts, Venues & Music (set 5) 1 passed8 failed

11.1%

IN291 227 OUT6 521

×

SimpleCircleDimensionsRadius5BFCL/simple_python_272 (calculate_circle_dimensions): radius=5.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 212 OUT407

✓

SimpleDrawRectangleRedBFCL/simple_python_261 (draw_rectangle): width=20, height=10, color=red.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 416 OUT99

×

SimpleFindFluteYamahaBFCL/simple_python_282 (find_flute): brand=Yamaha, specs=[open hole, C foot, silver headjoint].1 AI query 25 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 25

Failed

IN134 571 OUT2 326

×

SimpleGenerateScaleProgressionCBFCL/simple_python_296 (music_generator.generate_scale_progression): key=C, tempo=80, duration=4, scale_type=major or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 065 OUT473

×

SimpleInstrumentDetailsPianoYamahaBFCL/simple_python_278 (get_instrument_details): instrument=piano, manufacturer=Yamaha, features=[price,rating] (either order).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 099 OUT487

×

SimpleMetTopArtworksBFCL/simple_python_275 (metropolitan_museum.get_top_artworks): number=5, sort_by=popularity or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN20 858 OUT751

×

SimpleMixPaintYellowBlueBFCL/simple_python_258 (mix_paint_color): color1=yellow, color2=blue, lightness=60.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 794 OUT170

×

SimplePapalBiographyInnocentIIIBFCL/simple_python_255 (religious_history.get_papal_biography): papal_name=Innocent III, include_contributions=true.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN47 240 OUT1 183

×

SimpleSculptureLocatorMichelangeloBFCL/simple_python_268 (sculpture_locator.find_by_artist): artist=Michelangelo, material=Marble, location=Rome.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN20 972 OUT625

Tool Calling - Simple: Arts, Venues & Music (set 6) 1 passed6 failed

14.3%

IN145 541 OUT4 535

×

SimpleAnalyzeStructureB1004BFCL/simple_python_271 (analyze_structure): building_id=B1004, floors=[2,3,4], mode=dynamic.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 262 OUT477

×

SimpleConcertBookingEminemBFCL/simple_python_288 (concert_booking.book_ticket): artist=Eminem, city=New York City, num_tickets=2.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN31 499 OUT600

×

SimpleFindExhibitionNyBFCL/simple_python_267 (find_exhibition): location='New York City, NY', art_form=sculpture, month omitted, user_ratings=high.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 945 OUT329

×

SimpleInstrumentInfoViolin1721BFCL/simple_python_281 (get_instrument_info): name=Violin, maker=Stradivarius, year=1721.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN16 051 OUT632

×

SimpleInstrumentPriceStratocasterBFCL/simple_python_279 (instrument_price.get): brand=Fender, model='American Professional II Stratocaster', finish=Rosewood.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN27 476 OUT1 982

✓

SimpleModifyPaintingOilBFCL/simple_python_262 (modify_painting): size='12x18', medium=oil, dominant_color=red.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 987 OUT97

×

SimpleSongLyricsBohemianBFCL/simple_python_295 (get_song_lyrics): song_title='Bohemian Rhapsody', artist_name=Queen, lang=English or omitted.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN23 321 OUT418

Tool Calling - Simple: Food, Travel & Shopping (set 1) 1 passed7 failed

12.5%

IN249 648 OUT5 049

×

SimpleCookingConvertPoundsToOuncesBFCL/simple_python_365 (cooking_conversion.convert): 2 pounds->ounces of butter.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 496 OUT268

×

SimpleCurrencyExchangeUsdToCadBFCL/simple_python_388 (currency_exchange.convert): USD->CAD, amount=500.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN52 726 OUT868

×

SimpleGroceryStoreOrganicHoustonBFCL/simple_python_369 (grocery_store.find_nearby): Houston, categories {Organic,Fruits,Vegetables} any order.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN44 944 OUT726

×

SimpleHotelBookPetFriendlyManhattanBFCL/simple_python_380 (hotel_booking): Manhattan, single, 3 nights, 2023-03-10, pet_friendly.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 793 OUT495

×

SimpleMapsDistanceEiffelLouvreBFCL/simple_python_394 (maps.get_distance_duration): Eiffel Tower -> Louvre Museum, traffic omitted/false.1 AI query 8 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN33 596 OUT836

×

SimpleRecipeFindByDietMealBFCL/simple_python_353 (find_recipes): diet=gluten-free, meal_type=dinner, ingredients omitted.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN42 435 OUT902

×

SimpleRestaurantFinderItalianNYCBFCL/simple_python_361 (restaurant_finder): city=NYC, cuisine=Italian, diet=Gluten-free.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN38 186 OUT833

✓

SimpleTimeZoneLondon24hBFCL/simple_python_376 (time_zone_converter): city=London, country=UK, display_format=24h.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 472 OUT121

Tool Calling - Simple: Food, Travel & Shopping (set 2) 9 failed

0.0%

IN679 785 OUT10 985

×

SimpleBestSushiPlacesTokyoBFCL/simple_python_362 (get_best_sushi_places): city=Tokyo, top=5, review_rate=4.0.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN381 838 OUT4 097

×

SimpleCurrencyConverterUsdToGbpBFCL/simple_python_389 (currency_converter): USD->GBP, amount=200.0.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 618 OUT266

×

SimpleCurrentTimeSydneyCityBFCL/simple_python_377 (get_current_time): city=Sydney, country=Australia, format omitted/HH:MM:SS.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 496 OUT1 491

×

SimpleDistanceCalcNyToBostonTerrainBFCL/simple_python_397 (distance_calculator.calculate): New York -> Boston, consider_terrain=true.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN27 115 OUT641

×

SimpleHiltonAvailabilityParisBFCL/simple_python_381 (hilton_hotel.check_availability): Paris, 2023-04-04..08, 2 adults, chain Hilton/omitted.1 AI query 15 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN78 721 OUT1 706

×

SimpleRecipeCaloriesBeefLasagnaBFCL/simple_python_355 (recipe_info.get_calories): website=Foodnetwork.com, recipe=Beef Lasagna, meal_time omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 580 OUT308

×

SimpleRecipeUnitTbspToTspBFCL/simple_python_366 (recipe.unit_conversion): 2 tablespoon->teaspoon, precision omitted/1.1 AI query 15 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN73 897 OUT1 177

×

SimpleSafewayOrderPaloAltoBFCL/simple_python_370 (safeway.order): Palo Alto, [olive oil, rice], [3,1]. EXPECTED MODEL MISS (kept, do not loosen): "a five pound bag of rice" = quantity 1 (5 is the weight, not the count); models tend to send quantity 5. This is a genuine model error the test correctly catches.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 675 OUT315

×

SimpleVeganSoupRecipeUnder30BFCL/simple_python_354 (get_vegan_recipe): dish_type=soup, cooking_time=30, ingredient_preference omitted.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN55 845 OUT984

Tool Calling - Simple: Food, Travel & Shopping (set 3) 8 failed

0.0%

IN219 540 OUT4 884

×

SimpleBookHotelHiltonChicagoBFCL/simple_python_382 (book_hotel): Hilton Hotel, Chicago, single, 2022-12-10, 2 nights.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 040 OUT258

×

SimpleCookingTimeRoastChickenBFCL/simple_python_368 (calculate_cooking_time): weight_kg=1.5, method omitted/roast, temp omitted/180.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 112 OUT463

×

SimpleCurrencyConvertEurToCadBFCL/simple_python_390 (currency_conversion.convert): 150 EUR->CAD.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 137 OUT388

×

SimpleCurrentTimeSydneyLocationBFCL/simple_python_379 (get_current_time): location=Sydney, country=Australia, timezone omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 428 OUT201

×

SimpleParkingLotCentralParkBFCL/simple_python_395 (parking_lot.find_nearest): Central Park NY, radius=2, type omitted/public.1 AI query 11 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN48 158 OUT783

×

SimpleRecipeFinderVeganServes2BFCL/simple_python_356 (recipe_finder.find): servings=2, diet=vegan, prep_time=30.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN52 999 OUT1 736

×

SimpleRestaurantClosestSushiBostonBFCL/simple_python_363 (restaurant_search.find_closest): Boston, Sushi, amenities=[Patio].1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 556 OUT366

×

SimpleWholeFoodsPriceLABFCL/simple_python_371 (whole_foods.check_price): Los Angeles, [tomatoes, lettuce].1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN31 110 OUT689

Tool Calling - Simple: Food, Travel & Shopping (set 4) 7 failed

0.0%

IN380 284 OUT7 970

×

SimpleBookRoomThePlazaBFCL/simple_python_383 (book_room): The Plaza, single, 2 nights.1 AI query 20 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 20

Failed

IN93 593 OUT1 675

×

SimpleConvertCurrencyJpyToUsdBFCL/simple_python_393 (convert_currency): 20000 JPY->USD.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN12 540 OUT374

×

SimpleFindRestaurantItalianBrooklynBFCL/simple_python_364 (find_restaurant): Brooklyn, Italian, Gluten-free.1 AI query 16 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN70 450 OUT1 291

×

SimpleGetRecipeVeganChocCakeBFCL/simple_python_357 (get_recipe): dish_name=(vegan) chocolate cake, diet_preference=vegan.1 AI query 19 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 19

Failed

IN86 252 OUT1 921

×

SimpleHospitalLocateDenverBFCL/simple_python_396 (hospital.locate): Denver CO, radius=5, department=Pediatrics.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN27 468 OUT690

×

SimpleTimezoneConvertNyToLondonBFCL/simple_python_378 (timezone.convert): 3pm, New York -> London.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN45 039 OUT1 057

×

SimpleWholeFoodsTopBrandsBananasBFCL/simple_python_372 (whole_foods.find_top_brands): product=bananas, number omitted/5, organic=true.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN44 942 OUT962

Tool Calling - Simple: Food, Travel & Shopping (set 5) 1 passed5 failed

16.7%

IN264 437 OUT6 090

×

SimpleExchangeRateFeeGbpJpyBFCL/simple_python_391 (get_exchange_rate_with_fee): GBP->JPY, fee=0.02.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 034 OUT343

✓

SimpleHotelBookingBookParisBFCL/simple_python_384 (hotel_booking.book): Paris, 07-10-2022..07-20-2022, 2 adults, 1 child, Standard/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 963 OUT191

×

SimpleMuseumHoursMetSaturdayBFCL/simple_python_398 (get_museum_hours): museum=Metropolitan Museum of Art/The Met, day=Saturday.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN22 957 OUT428

×

SimpleRecipeSearchGfCookieBFCL/simple_python_358 (recipe_search): diet=[gluten free], time_limit=30, dish=cookie.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN39 775 OUT1 086

×

SimpleRestaurantSearchRatedNYCBFCL/simple_python_399 (restaurant_search): NYC, Italian, rating=4, accepts_credit_cards=true.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN27 332 OUT875

×

SimpleWalmartPurchaseSanJoseBFCL/simple_python_373 (walmart.purchase): San Jose, [apples, rice, (bottled) water], pack_size=[1,1,12].1 AI query 28 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 28

Failed

IN159 376 OUT3 167

Tool Calling - Simple: Food, Travel & Shopping (set 6) 5 failed

0.0%

IN97 175 OUT3 347

×

SimpleGameHighestScoringPlayerBFCL/simple_python_350 (get_highest_scoring_player): game=Valorant, season=2022.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN11 616 OUT572

×

SimpleHotelBookRoomKingLABFCL/simple_python_385 (hotel_bookings.book_room): LA, king size, 15-10-2023, 2 nights, rooms omitted/1.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN11 826 OUT349

×

SimpleLatestExchangeGbpJpyBFCL/simple_python_392 (latest_exchange_rate): GBP->JPY, amount omitted/1.0.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN18 203 OUT392

×

SimpleNutritionalInfoAvocadoBFCL/simple_python_374 (grocery_info.nutritional_info): Walmart, avocado, [Protein,Calories,Carbohydrates].1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN11 702 OUT414

×

SimpleRecipeSearchVegPastaBFCL/simple_python_359 (recipe_search): dietary_restriction=Vegetarian, ingredients=[pasta,cheese], servings=2.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN43 828 OUT1 620

Tool Calling - Simple: Food, Travel & Shopping (set 7) 4 failed

0.0%

IN75 592 OUT2 253

×

SimpleBookHotelLuxuryParadiseBFCL/simple_python_386 (book_hotel): Hotel Paradise, Las Vegas, luxury, 05-12-2022, 3 days, view=city.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN8 138 OUT307

×

SimpleFindRecipeCaloriesCarbonaraBFCL/simple_python_360 (find_recipe): recipeName=pasta carbonara, maxCalories=500.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 908 OUT704

×

SimpleGameMultiplayerFinderBFCL/simple_python_351 (multiplayer_game_finder): platform=Windows 10, rating=4.5, genre omitted/Action.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN24 293 OUT629

×

SimpleWalmartPriceCheckPumpkinsBFCL/simple_python_375 (walmart.check_price): [pumpkins,eggs], quantities [3,24] or [3,2], store omitted.1 AI query 7 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN25 253 OUT613

Tool Calling - Simple: Food, Travel & Shopping (set 8) 1 passed2 failed

33.3%

IN50 882 OUT1 584

×

SimpleFindRecipeVeganBrowniesBFCL/simple_python_367 (find_recipe): dietary_restrictions=vegan, recipe_type=dessert (brownies), time=30.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN26 468 OUT829

×

SimpleGameAverageUserScoreBFCL/simple_python_352 (gamespot.getAverageUserScore): game=Zelda BotW, platform=Nintendo Switch/all platforms.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN19 518 OUT555

✓

SimpleHotelBookingPlazaRangeBFCL/simple_python_387 (hotel_booking): Plaza Hotel, New York NY, 2022-06-01..2022-06-04, rooms omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 896 OUT200

Tool Calling - Simple: Knowledge, Places & History (set 1) 10 failed

0.0%

IN635 627 OUT9 835

×

SimpleAirQualityChicagoBFCL/simple_python_203 (get_air_quality): location=Chicago, detail=true.1 AI query 10 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN44 756 OUT751

×

SimpleEvolutionScientistBFCL/simple_python_242 (get_scientist_for_discovery): discovery=Theory of Evolution.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 217 OUT314

×

SimpleGermanWarEventsBFCL/simple_python_231 (history.get_key_events): country=Germany/DE, 1871-1945, event_type=[War].1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 535 OUT889

×

SimpleKingOfFranceBFCL/simple_python_230 (historic_leader_search): location=France, date=1510, title omitted or King.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 380 OUT293

×

SimplePandaPopulationBFCL/simple_python_201 (estimate_population): species=panda(s), country=China/CN, year omitted or 2024.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN360 976 OUT3 962

×

SimplePersonalityTraitsTypeBFCL/simple_python_227 (get_personality_traits by type): type=ENFJ, traits=[strengths, weaknesses].1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN22 579 OUT713

×

SimplePresidentCivilWarBFCL/simple_python_238 (us_history.get_president): event=American Civil War, year in 1861-1865.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN40 290 OUT669

×

SimpleRestaurantNearbyBFCL/simple_python_204 (restaurant.find_nearby): location=Seattle[, WA], cuisine=Chinese, max_distance=10.1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN78 544 OUT1 290

×

SimpleTordesillasDateBFCL/simple_python_233 (european_history.get_event_date): event=Treaty of Tordesillas, format=YYYY.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 592 OUT398

×

SimpleTrafficBostonNewYorkBFCL/simple_python_205 (get_traffic_info): start=Boston, end=New York/NYC, mode omitted or driving.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN26 758 OUT556

Tool Calling - Simple: Knowledge, Places & History (set 2) 1 passed9 failed

10.0%

IN286 618 OUT5 821

×

SimpleBitcoinNewsBFCL/simple_python_210 (get_news): topic=Bitcoin, quantity=5, region omitted or US.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN30 924 OUT526

×

SimpleEuropeanWars19cBFCL/simple_python_234 (history_eu.fetch_events): century=19, region=any of N/S/E/W, category=Wars. EXPECTED MODEL MISS (kept, do not loosen): region is an enum (Northern/Southern/Eastern/Western); the prompt says only "European", so any of the four is accepted, but models often send the invalid value "Europe". Failing to respect the enum is a genuine model error the test correctly catches.1 AI query 13 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 13

Failed

IN61 035 OUT1 487

×

SimpleLibraryNearbyBFCL/simple_python_209 (public_library.find_nearby): location=Boston MA/Massachusetts, facilities=[Fiction, Wi-Fi] (any order).1 AI query 17 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 17

Failed

IN85 961 OUT1 530

×

SimpleMonarchOfEnglandBFCL/simple_python_232 (monarch.getMonarchOfYear): location=England/ENG, year=1800, fullName=true.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN22 416 OUT425

×

SimpleNeutronDiscovererBFCL/simple_python_243 (get_discoverer): discovery=neutron, detail=true.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 146 OUT405

×

SimpleParksNearbyBFCL/simple_python_206 (parks.find_nearby): location=London[, UK], amenities=[Tennis Court].1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 484 OUT495

×

SimplePersonalityTraitsHobbyBFCL/simple_python_228 (get_personality_traits by hobby): hobby=jogging, trait_count=3.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 274 OUT313

×

SimplePopulationGrowthBFCL/simple_python_221 (population_growth_estimate): location=London, years=5, rate omitted or 1.2.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 260 OUT205

✓

SimplePresidentDuringEventBFCL/simple_python_241 (US_President_During_Event): event=Civil War, country omitted or USA.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 889 OUT162

×

SimpleTreatyLisbonDateBFCL/simple_python_235 (get_event_date): event=Treaty of Lisbon (variants), location omitted or Lisbon[, Portugal].1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 229 OUT273

Tool Calling - Simple: Knowledge, Places & History (set 3) 2 passed8 failed

20.0%

IN860 384 OUT10 801

×

SimpleAppleStockInfoBFCL/simple_python_212 (get_stock_info): company=Apple Inc., detail_level=detailed, market omitted or NASDAQ.1 AI query 21 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 21

Failed

IN111 669 OUT1 416

×

SimpleCivilWarStartDateBFCL/simple_python_236 (us_history.get_event_info): event=American Civil War/Civil War, specific_info=Start Date.1 AI query 30 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 30

Failed

IN182 162 OUT2 354

✓

SimpleFlightBookBFCL/simple_python_213 (flight.book): departure=San Francisco/SF, destination=London, date=2022-04-27 (variants), time=afternoon, direct=true.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 278 OUT182

×

SimpleGravitationPublicationYearBFCL/simple_python_244 (publication_year.find): author=Isaac Newton, work=Law of Universal Gravitation (variants), location omitted or all.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 024 OUT176

×

SimpleMovieBriefBFCL/simple_python_215 (movie_details.brief): title=Interstellar, extra_info omitted or false.1 AI query 22 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 22

Failed

IN117 582 OUT1 477

×

SimplePresidentIn1861BFCL/simple_python_239 (US_president.in_year): year=1861, full_name omitted or true.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 030 OUT435

×

SimpleRadiumDiscovererBFCL/simple_python_245 (discoverer.get): element_name=radium (quote variants), year omitted or 0, first omitted or true.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN365 308 OUT3 733

✓

SimpleSendEmailBFCL/simple_python_211 (send_email): to=john.doe@example.com, subject=Meeting, body (opt trailing period), cc/bcc empty.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 280 OUT107

×

SimpleShortestDistanceBFCL/simple_python_207 (calculate_shortest_distance): start=New York (variants), end=Miami (variants), route_preference=Shortest.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 181 OUT363

×

SimpleUpcomingConcertsBFCL/simple_python_214 (event_finder.find_upcoming): location=New York (variants), genre=Rock, days_ahead=30.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN27 870 OUT558

Tool Calling - Simple: Knowledge, Places & History (set 4) 10 failed

0.0%

IN364 454 OUT8 464

×

SimpleBmiCalcBFCL/simple_python_222 (calculate_bmi): weight=70, height=180, unit omitted or metric.1 AI query 9 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN41 816 OUT888

×

SimpleCarEmissionsBFCL/simple_python_200 (calculate_emissions): distance=12000, fuel_type=gas, fuel_efficiency omitted or 25.0, efficiency_reduction omitted or 0.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN42 213 OUT1 074

×

SimpleFmriAnalyzeBFCL/simple_python_217 (fMRI.analyze): data_source path, sequence_type=multi-band, smooth=6, voxel_size=2.1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN47 737 OUT875

×

SimpleGetDirectionsBFCL/simple_python_208 (map_service.get_directions): start=New York/NYC, end=Los Angeles/LA, avoid=[highways, tolls] (any order).1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN52 333 OUT1 246

×

SimpleGravityDiscoveryBFCL/simple_python_246 (science_history.get_discovery_details): discovery=Gravity, method_used omitted or default.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 118 OUT487

×

SimpleGroupDynamicsBFCL/simple_python_223 (group_dynamics.pattern): total=50, extroverts=15, introverts=35.1 AI query 9 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN41 991 OUT998

×

SimpleMostFollowedBFCL/simple_python_224 (social_media_analytics.most_followed): topic=psychology, sub_topics=[behaviour, group dynamics], region omitted or all.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN32 443 OUT718

×

SimpleNeuronCoordinatesBFCL/simple_python_219 (get_neuron_coordinates): neuron_type=GABA, brain_region=All/all part of the brain/entire brain.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN52 792 OUT1 513

×

SimplePresidentIn1940BFCL/simple_python_240 (history_api.get_president_by_year): year=1940, full_term_only omitted/true/false.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 801 OUT296

×

SimpleSentimentAnalysisBFCL/simple_python_216 (sentiment_analysis): text=the review, language=english/en.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 210 OUT369

Tool Calling - Simple: Knowledge, Places & History (set 5) 10 failed

0.0%

IN736 952 OUT14 959

×

SimpleBigFiveScoresBFCL/simple_python_229 (get_bigfive_scores): characteristics=[efficient, organized, easy going, compassionate], scale omitted or medium.1 AI query 39 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 39

Failed

IN269 206 OUT3 833

×

SimpleChristianityHistoryBFCL/simple_python_249 (religion.history_info): religion=Christianity, till_century=14, include_people omitted or false.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN52 828 OUT2 013

×

SimpleEinsteinContributionBFCL/simple_python_247 (historical_contrib.get_contrib): scientist=Albert Einstein, date=1915-03-17 (variants), category omitted or all.1 AI query 27 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 27

Failed

IN161 798 OUT2 607

×

SimpleHistoricalGdpBFCL/simple_python_237 (get_historical_GDP): country=United States/US, start_year=1960, end_year=2000.1 AI query 13 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 13

Failed

IN64 417 OUT1 264

×

SimpleMriReportBFCL/simple_python_218 (patient.get_mri_report): patient_id=546382, mri_type omitted or brain, status=concluded.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 391 OUT368

×

SimpleNeuronalActivityBFCL/simple_python_220 (calculate_neuronal_activity): input_synaptic_rate=200, weight=0.5, decay_rate=0.1.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN48 056 OUT928

×

SimpleReadingPreferenceBFCL/simple_python_225 (psych_research.get_preference): category=reading, option_one=digital[reading], option_two=physical book(s), demographic omitted or all.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN37 772 OUT1 532

×

SimpleRelativityInventionBFCL/simple_python_248 (science_history.get_invention): invention_name=theory of relativity, want_year=true.1 AI query 8 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN37 359 OUT596

×

SimpleRenewableEmissionSavingsBFCL/simple_python_202 (calculate_emission_savings): energy_type=renewable, usage_duration=3, region=California/CA. EXPECTED MODEL MISS (kept, do not loosen): the prompt says "renewable energy sources" generically; models often over-specify a concrete type (e.g. "solar"). Ground truth wants "renewable".1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN18 051 OUT1 287

×

SimpleZodiacCompatibilityBFCL/simple_python_226 (get_zodiac_compatibility): sign1=Aries, sign2=Gemini, scale omitted or percentage.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN28 074 OUT531

Tool Calling - Simple: Law, Finance & Records (set 1) 10 failed

0.0%

IN479 554 OUT10 845

×

SimpleAirQualityByDateBFCL/simple_python_196 (air_quality): location London, date reformatted to 08-16-2022.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 086 OUT388

×

SimpleCivilCaseDetailsWithDissentBFCL/simple_python_167 (law.civil.get_case_details): case_title Roe v. Wade, include_dissent true.1 AI query 27 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 27

Failed

IN158 720 OUT2 540

×

SimpleCompoundedInterestQuarterlyBFCL/simple_python_153 (calculate_compounded_interest): principal 5000, rate 0.03, time 5, n 4 (quarterly).1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN33 046 OUT844

×

SimpleCrimeRateByCityBFCL/simple_python_164 (get_crime_rate): city San Francisco, state California/CA, type violent/omitted, year 2020.1 AI query 13 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 13

Failed

IN64 459 OUT1 688

×

SimpleCriminalRecordsByPersonBFCL/simple_python_158 (get_criminal_records): name Mr. X, location New York, NY, from_year 2012, to_year 2015.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN33 172 OUT975

×

SimpleDetailedWeatherForecastHoursBFCL/simple_python_185 (detailed_weather_forecast): location New York, duration 72 hours, include_precipitation true.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN42 406 OUT748

×

SimpleFutureValueByDurationBFCL/simple_python_155 (calculate_future_value): initial_investment 1000, interest_rate 0.05, duration 2, compounded omitted or 1.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 386 OUT459

×

SimpleLawsuitDetailsWithExtraFieldsBFCL/simple_python_178 (get_lawsuit_details): case_number LAX2019080202, court_location Los Angeles, additional_details omitted or full default list.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN32 808 OUT540

×

SimpleLawsuitSearchByFilingDateBFCL/simple_python_168 (lawsuit_search): company Google/GOOG, start_date 01-01-2021, location California, status ongoing/omitted.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN48 079 OUT1 046

×

SimpleLawyerExperienceMonthsBFCL/simple_python_175 (lawyer.get_experience): name John Doe, law_type Bankruptcy.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN32 392 OUT1 617

Tool Calling - Simple: Law, Finance & Records (set 2) 10 failed

0.0%

IN816 001 OUT12 827

×

SimpleAirQualityIndexByTimeBFCL/simple_python_197 (get_air_quality_index): location San Diego, time 12pm/12:00.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 606 OUT238

×

SimpleCivilCasesRetrieveTheftBFCL/simple_python_165 (civil_cases.retrieve): year 2020, crime_type theft, location Los Angeles.1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 18

Failed

IN92 665 OUT1 581

×

SimpleFindNearbyDivorceLawyerBFCL/simple_python_166 (lawyer.find_nearby): city Chicago IL, specialty [Divorce], fee 400. NOTE: single-call contract kept intentionally; the model occasionally emits a spurious second call (no other visible tool fits this query), which is a model-reliability signal, not a test defect.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN51 388 OUT1 129

×

SimpleGenerateLawContractRentalBFCL/simple_python_162 (generate_law_contract): parties [John, Alice], contract_type Rental Agreement, location California/CA.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN51 707 OUT877

×

SimpleLawsuitDetailsFindByCompanyBFCL/simple_python_176 (lawsuit_details.find): company_name Apple Inc., year 2010, case_type Patent/IPR.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN410 645 OUT4 789

×

SimpleLawsuitDetailsWithVerdictBFCL/simple_python_181 (get_lawsuit_details): case_number 123456-ABC, court_location Los Angeles, with_verdict true.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN27 472 OUT479

×

SimpleLocalNurseryAnnualPlantsBFCL/simple_python_193 (local_nursery.find): location Toronto, plant_types [Annual].1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN79 948 OUT1 671

×

SimplePropertyRecordsWithOwnerBFCL/simple_python_163 (property_records.get): address 123 main street, parcel_number 1234567890, county Santa Clara, include_owner true.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN37 322 OUT828

×

SimpleRoiSavingsAccountBFCL/simple_python_150 (calculate_roi): deposit 1000, annual_interest_rate 0.03, years 1.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 021 OUT441

×

SimpleWeatherForecastDetailedDaysBFCL/simple_python_189 (weather_forecast_detailed): location New York, days 3, details true.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN27 227 OUT794

Tool Calling - Simple: Law, Finance & Records (set 3) 10 failed

0.0%

IN929 485 OUT16 848

×

SimpleCaseInfoVictimBFCL/simple_python_160 (get_case_info): docket 2022/AL2562, court California/CA, info_type victim.1 AI query 46 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 46

Failed

IN346 988 OUT4 752

×

SimpleCrimeStatuteTheftBFCL/simple_python_161 (crime_statute_lookup): jurisdiction California/CA, crime theft, detail_level detailed.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 234 OUT1 541

×

SimpleCurrentWeatherCelsiusBFCL/simple_python_186 (current_weather_condition): city Tokyo, country Japan, measurement c/omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 951 OUT1 343

×

SimpleEnvironmentalAirQualityDaysBFCL/simple_python_199 (environmental_data.air_quality_index): location San Jose, days 3.1 AI query 31 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 31

Failed

IN186 170 OUT2 516

×

SimpleFelonyCrimeRecordBFCL/simple_python_156 (crime_record.get_record): case_number CA123456, county San Diego, details true.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN52 331 OUT1 151

×

SimpleHighestGrossingBankBFCL/simple_python_151 (highest_grossing_banks): country U.S variants, year 2020, top_n 1.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN37 049 OUT571

×

SimpleHistoricalFraudCaseBFCL/simple_python_170 (law_case_search.find_historical): subject fraud, from_year 2010, to_year 2015.1 AI query 18 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 18

Failed

IN93 446 OUT1 971

×

SimpleLawsuitCasesByCompanyYearBFCL/simple_python_177 (get_lawsuit_cases): company_name Facebook, year 2018, status all/omitted. EXPECTED MODEL MISS (kept, do not loosen): "Patent" is the case TYPE, not the status (enum is open/closed/all); status should be omitted/"all". Models often wrongly set status="Patent".1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN51 902 OUT1 132

×

SimpleLawsuitInfoByCaseNumberBFCL/simple_python_182 (lawsuit_info): case_number XYZ123, year omitted/2023, location omitted/all.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN45 944 OUT555

×

SimplePlantsForSlopeErosionBFCL/simple_python_194 (get_plants_for_slope): slope_type hill/steep/moderate, num_results 3.1 AI query 15 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN73 470 OUT1 316

Tool Calling - Simple: Law, Finance & Records (set 4) 10 failed

0.0%

IN488 270 OUT9 725

×

SimpleActDetailsAmendmentBFCL/simple_python_159 (get_act_details): act_name Criminal Law Amendment Act, amendment_year 2013.1 AI query 20 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 20

Failed

IN107 138 OUT2 029

×

SimpleCheckFeloniesByNameBFCL/simple_python_157 (criminal_history.check_felonies): full_name John Doe, birth_date 01-01-1980, state California/CA.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN38 123 OUT788

×

SimpleCourtCaseSearchNoFullTextBFCL/simple_python_169 (court_case.search): docket_number 123456, location Texas, full_text false/omitted.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN28 284 OUT446

×

SimpleCurrentWeatherTempHumidityBFCL/simple_python_187 (get_current_weather): location Seattle, include_temperature true/omitted, include_humidity true/omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 156 OUT250

×

SimpleLawsuitCheckClosedStatusBFCL/simple_python_184 (lawsuit.check_case): case_id 1234, closed_status true.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 321 OUT788

×

SimpleLawsuitsSearchByCompanyLocationBFCL/simple_python_180 (lawsuits_search): company_name Google, location California/CA, year 2020, case_type all/omitted.1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 18

Failed

IN93 127 OUT1 430

×

SimpleMutualFundBalanceBFCL/simple_python_152 (calculate_mutual_fund_balance): investment_amount 50000, annual_yield 0.05, years 3.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN28 410 OUT643

×

SimpleSlopeGradientCoordinatesBFCL/simple_python_192 (calculate_slope_gradient): point1 [40.7128,-74.006], point2 [34.0522,-118.2437], unit degree/omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 494 OUT513

×

SimpleStateLawCaseYearRangeBFCL/simple_python_173 (law_case_search): topic land disputes, year_range [2015,2021], location New York, judicial_system state.1 AI query 17 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 17

Failed

IN119 550 OUT2 077

×

SimpleTopConstitutionalCasesBFCL/simple_python_174 (get_top_cases): field_of_law constitutional law, top_number 10, country China/CN.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 667 OUT761

Tool Calling - Simple: Law, Finance & Records (set 5) 10 failed

0.0%

IN423 204 OUT9 479

×

SimpleCarbonFootprintLifestyleBFCL/simple_python_195 (calculate_carbon_footprint): daily_miles 20, meat_meals_per_week 3, annual_trash_weight 500, flights_per_year omitted/0.1 AI query 21 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 21

Failed

IN112 333 OUT1 879

×

SimpleDailyWaterIntakeWeightBFCL/simple_python_198 (calculate_daily_water_intake): weight 70, activity_level omitted/moderate, climate omitted/temperate.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 926 OUT715

×

SimpleFetchLawCaseByNumberBFCL/simple_python_171 (fetch_law_case_details): case_number 43403, court New York, year 2018.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 319 OUT439

×

SimpleFutureValueByPresentValueBFCL/simple_python_154 (calculate_future_value): present_value 5000, annual_interest_rate 0.05, years 10, compounds_per_year omitted/1.1 AI query 19 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 19

Failed

IN102 927 OUT1 845

×

SimpleHumidityForecastDaysBFCL/simple_python_188 (weather.humidity_forecast): location Miami, days 7, min_humidity omitted/0.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 885 OUT311

×

SimpleLatestCourtCaseCompaniesBFCL/simple_python_179 (find_latest_court_case): company1 Apple, company2 Samsung, country USA/omitted.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN36 900 OUT609

×

SimpleLawsuitSearchByCountyBFCL/simple_python_183 (lawsuit_search): entity Apple, county Santa Clara County, state California/omitted.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN57 094 OUT933

×

SimpleLegalCaseFetchDetailsBFCL/simple_python_172 (legal_case.fetch): case_id R vs Adams, details true.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN18 864 OUT1 358

×

SimpleParkElevationAreaBFCL/simple_python_190 (park_information): park_name Yellowstone, information [Elevation, Area] (order-independent).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 953 OUT445

×

SimpleTallestMountainsRadiusBFCL/simple_python_191 (locate_tallest_mountains): location Denver, radius 50, amount 5.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 003 OUT945

Tool Calling - Simple: Live Answer Tool (isolated) (set 1) 1 passed

100.0%

IN4 013 OUT101

✓

SimpleAnswerStringLogistic1BFCL/live_simple_32-9-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L113: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 013 OUT101

Tool Calling - Simple: Live Answer Tool (isolated) (set 2) 1 failed

0.0%

IN83 248 OUT1 855

×

SimpleAnswerStringLogistic2BFCL/live_simple_33-10-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L114: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.1 AI query 22 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 22

Failed

IN83 248 OUT1 855

Tool Calling - Simple: Live Answer Tool (isolated) (set 3) 1 skipped

-

IN0 OUT0

–

SimpleAnswerStringValidBFCL/live_simple_34-11-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L115: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Harness-semantic mismatch (item flaw for our runner): the query ('...Valid?') is directly answerable, and our system prompt permits answering in plain text when no tool is needed, so the model reasonably returns 0 tool calls instead of routing to the answer_string meta-tool. Not a deterministic tool-routing test. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Simple: Live Answer Tool (isolated) (set 4) 1 passed

100.0%

IN4 050 OUT218

✓

SimpleAnswerStringWhatIsBFCL/live_simple_35-12-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L116: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 050 OUT218

Tool Calling - Simple: Live Assistants, Weather & Devices (set 1) 1 passed2 failed

33.3%

IN45 228 OUT973

×

SimpleChangeFoodBurgerBFCL/live_simple_20-4-0 (change_food): food_item='burger', modification_request='no onions, extra cheese'.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN17 336 OUT301

✓

SimpleThinqAirCleanHighBFCL/live_simple_40-17-0 (ThinQ_Connect): AIR_CLEAN mode, HIGH wind, monitoring on, air clean POWER_ON. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. windStrength broadened (high|strongest|strong) for "strongest wind strength". EXPECTED MODEL MISS: 27B sets airConJobMode=AIR_CLEAN but omits airCleanOperationMode=POWER_ON — the prompt asks to "start the air purifying function", so POWER_ON is the fair GT; kept strict (a real tool-use miss).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 956 OUT145

×

SimpleWeatherTelAvivBFCL/live_simple_4-3-0 (get_current_weather): location='Tel Aviv, Israel', unit=fahrenheit or omitted.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN22 936 OUT527

Tool Calling - Simple: Live Assistants, Weather & Devices (set 10) 1 passed2 failed

33.3%

IN262 852 OUT3 753

×

SimpleChafodOrderPizzaBFCL/live_simple_23-5-1 (ChaFod): TheFod='PIZZA'.1 AI query 12 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN40 600 OUT804

✓

SimpleUberEatMcdonaldsPizzaBFCL/live_simple_29-7-2 (uber.eat.order): restaurant="McDonald's", items=[pizza], quantities=[1].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 664 OUT97

×

SimpleWeatherShanghaiBFCL/live_simple_13-3-9 (get_current_weather): location='Shanghai, China', unit=celsius. Prompt is Chinese, copied verbatim.1 AI query 40 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 40

Failed

IN217 588 OUT2 852

Tool Calling - Simple: Live Assistants, Weather & Devices (set 11) 3 failed

0.0%

IN142 765 OUT3 510

×

SimpleChafodSwitchBurgerBFCL/live_simple_24-5-2 (ChaFod): TheFod='BURGER'.1 AI query 20 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 20

Failed

IN80 624 OUT1 137

×

SimpleFindBeerSierraNevadaBFCL/live_simple_48-21-0 (find_beer): brewery='Sierra Nevada', taste=bitter, aroma=hoppy, color=pale, style=lager; defaults omitted.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN45 407 OUT1 145

×

SimpleWeatherChennaiBFCL/live_simple_15-3-11 (get_current_weather): location='Chennai, Tamil Nadu', unit=fahrenheit or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN16 734 OUT1 228

Tool Calling - Simple: Live Assistants, Weather & Devices (set 12) 3 failed

0.0%

IN41 916 OUT1 765

×

SimpleChafodGreensSaladBFCL/live_simple_25-5-3 (ChaFod): TheFod='SALAD' (greens mapped to salad).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 712 OUT109

×

SimpleFindBeerCreekPorterBFCL/live_simple_49-21-1 (find_beer): brewery='Creek', taste='slightly bitter', aroma=chocolatey, color=dark, style=porter; defaults omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN17 351 OUT619

×

SimpleWeatherQuintanaRooBFCL/live_simple_19-3-15 (get_current_weather): location='Quintana Roo, Mexico', unit=celsius.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN16 853 OUT1 037

Tool Calling - Simple: Live Assistants, Weather & Devices (set 13) 1 passed1 failed

50.0%

IN20 987 OUT1 275

✓

SimpleLexListExportsV2BFCL/live_simple_30-8-0 (aws.lexv2_models.list_exports): botId='my-bot-id', botVersion='v2', sortBy=ASC or omitted, others omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 842 OUT241

×

SimpleWeatherLangSonBFCL/live_simple_16-3-12 (get_current_weather): location='Lang Son, Vietnam', unit=fahrenheit or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN16 145 OUT1 034

Tool Calling - Simple: Live Assistants, Weather & Devices (set 14) 2 failed

0.0%

IN29 714 OUT1 159

×

SimpleLexListExportsDescBFCL/live_simple_31-8-1 (aws.lexv2_models.list_exports): botId='B12345', botVersion='v1', sortBy=DESC, others omitted.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN19 690 OUT850

×

SimpleWeatherBostonBFCL/live_simple_17-3-13 (get_current_weather): location='Boston, MA', unit=fahrenheit or omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 024 OUT309

Tool Calling - Simple: Live Assistants, Weather & Devices (set 15) 1 passed1 failed

50.0%

IN11 073 OUT291

✓

SimpleGetUserInfoBlackBFCL/live_simple_0-0-0 (get_user_info): user_id=7890, special='black'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 369 OUT128

×

SimpleWeatherMoscowBFCL/live_simple_18-3-14 (get_current_weather): location='Moscow, Russia', unit=celsius.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN6 704 OUT163

Tool Calling - Simple: Live Assistants, Weather & Devices (set 16) 3 failed

0.0%

IN49 864 OUT1 734

×

SimpleFetchWeatherCoordsBFCL/live_simple_39-16-0 (fetch_weather_data): latitude=37.8651, longitude=-119.5383; url/units/language defaults omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN17 334 OUT829

×

SimpleGithubStarGorillaBFCL/live_simple_1-1-0 (github_star): repos='ShishirPatil/gorilla,gorilla-llm/gorilla-cli', aligned=true.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN16 379 OUT546

×

SimpleWeatherBangkokBFCL/live_simple_14-3-10 (get_current_weather): location='Bangkok, Thailand', unit=fahrenheit or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN16 151 OUT359

Tool Calling - Simple: Live Assistants, Weather & Devices (set 17) 1 passed2 failed

33.3%

IN54 824 OUT1 348

×

SimpleCarbonIntensityGBBFCL/live_simple_50-22-0 (get_latest_carbon_intensity): zone='Great Britain' or 'GB'.1 AI query 10 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN31 966 OUT654

✓

SimpleMultiplyThreeTwoBFCL/live_simple_47-20-0 (multiply): a=3, b=2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 596 OUT63

×

SimpleWeatherOpenMeteoMariposaBFCL/live_simple_38-15-0 (get_current_weather, open-meteo variant): location='Mariposa, CA', unit=fahrenheit; url default omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN18 262 OUT631

Tool Calling - Simple: Live Assistants, Weather & Devices (set 2) 1 passed2 failed

33.3%

IN22 744 OUT448

×

SimpleChangeFoodChickenSpicyBFCL/live_simple_21-4-1 (change_food): food_item='chicken dish', modification_request='extra spicy'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 565 OUT134

✓

SimpleThinqAirDryMidBFCL/live_simple_41-17-1 (ThinQ_Connect): AIR_DRY mode, MID wind (or omitted), all else optional. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. windStrength broadened (mid|medium) for "medium wind strength"; air-clean off-state accepts off/power_off.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 893 OUT78

×

SimpleWeatherDivinopolisBFCL/live_simple_5-3-1 (get_current_weather): location='Divinopolis, MG' (verbatim), unit=fahrenheit or omitted. Prompt is Portuguese, copied verbatim.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 286 OUT236

Tool Calling - Simple: Live Assistants, Weather & Devices (set 3) 1 passed2 failed

33.3%

IN61 855 OUT1 173

×

SimpleParseAnswerNoDefineBFCL/live_simple_36-13-0 (parseAnswer): answer='The text does not define logistic regression, therefore I cannot answer this question.'1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN37 403 OUT642

✓

SimpleThinqKoreanTarget22BFCL/live_simple_42-17-2 (ThinQ_Connect): coolTargetTemperature=22 required; mode COOL and wind MID optional. Prompt is Korean, copied verbatim. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. Prompt is dehumidify (제습) so GT's optional airConJobMode=COOL is a source oddity — accept cool/dehumidify/air_dry (optional). windStrength MID broadened (mid|medium). The "22도" target-temp is accepted in either coolTargetTemperature or targetTemperature (the two are genuinely ambiguous for "target temperature").1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 924 OUT113

×

SimpleWeatherSanFranciscoBFCL/live_simple_6-3-2 (get_current_weather): location='San Francisco, CA', unit=fahrenheit or omitted.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN19 528 OUT418

Tool Calling - Simple: Live Assistants, Weather & Devices (set 4) 1 passed2 failed

33.3%

IN46 287 OUT1 022

×

SimpleParseAnswerNoDefineValidityBFCL/live_simple_37-14-0 (parseAnswer): answer='The text does not define logistic regression, therefore I cannot answer this question.'1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 503 OUT199

✓

SimpleThinqCoolHighWindBFCL/live_simple_43-17-3 (ThinQ_Connect): HIGH wind required; COOL mode and coolTargetTemperature=24 optional. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. air-clean off-state broadened (power_off|off) — the model files the unrequested clean-mode as "off".1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 872 OUT90

×

SimpleWeatherRigaBFCL/live_simple_7-3-3 (get_current_weather): location='Riga, Latvia'; unit fahrenheit/celsius/omitted all accepted.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN33 912 OUT733

Tool Calling - Simple: Live Assistants, Weather & Devices (set 5) 1 passed2 failed

33.3%

IN26 497 OUT607

✓

SimpleThinqTurnedOnCoolBFCL/live_simple_44-18-0 (ThinQ_Connect): airConOperationMode=POWER_ON required; COOL currentJobMode optional. NOTE: tool flattened to nullable top-level params (see struct comment) so a valid partial call is one tool call, not a forced retry. windStrength/airClean off-state broadened to synonyms (mid|medium, stop|off|power_off).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 045 OUT74

×

SimpleUberComfortAddisonBFCL/live_simple_2-2-0 (uber.ride): loc='2020 Addison Street, Berkeley, CA, USA', type=comfort, time=600.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 856 OUT251

×

SimpleWeatherLondonBFCL/live_simple_8-3-4 (get_current_weather): location='London, UK' or 'London, England'; unit fahrenheit or omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 596 OUT282

Tool Calling - Simple: Live Assistants, Weather & Devices (set 6) 1 passed2 failed

33.3%

IN17 927 OUT370

×

SimpleThinqCoolAirCleanStartBFCL/live_simple_45-18-1 (ThinQ_Connect): airCleanOperationMode=START, airConOperationMode=POWER_ON, coolTargetTemperature=20. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. airCleanOperationMode "air cleaning on" broadened (start|on|power_on). airConOperationMode=POWER_ON and coolTargetTemperature=20 kept strict — both fair GT. EXPECTED MODEL MISS: 27B gets job-mode + air-clean START + temp, but omits airConOperationMode=POWER_ON.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.airConOperationMode?.ToLower()).AnyOf(["power_on"])

Failed

IN5 097 OUT93

✓

SimpleUberPlusBakerBFCL/live_simple_3-2-1 (uber.ride): loc='221B Baker Street, Berkeley, CA, USA', type=plus, time=600.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 109 OUT151

×

SimpleWeatherHyderabadBFCL/live_simple_9-3-5 (get_current_weather): location='Hyderabad, India', unit=fahrenheit or omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 721 OUT126

Tool Calling - Simple: Live Assistants, Weather & Devices (set 7) 2 passed1 failed

66.7%

IN17 959 OUT535

✓

SimpleThinqFlatCoolCleanBFCL/live_simple_46-19-0 (ThinQ_Connect, flat params): COOL, MID, monitoring on, clean START, POWER_ON, power-save off, target 24C.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 148 OUT189

✓

SimpleUberPlusVietnameseBFCL/live_simple_26-6-0 (uber.ride): loc Vietnamese/English form, type=plus, time=10 (minutes). Prompt is Vietnamese, copied verbatim.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 071 OUT171

×

SimpleWeatherYosemiteMariposaBFCL/live_simple_10-3-6 (get_current_weather): location='Mariposa, CA', unit=celsius.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 740 OUT175

Tool Calling - Simple: Live Assistants, Weather & Devices (set 8) 3 failed

0.0%

IN26 238 OUT641

×

SimpleThinqCoolScheduledBFCL/live_simple_51-23-0 (ThinQ_Connect): COOL, HIGH wind, POWER_ON, power-save on, target 22C, start in 1h30m. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. airConOperationMode=POWER_ON kept strict ("Start in 1h30m" schedules a power-on). EXPECTED MODEL MISS: 27B gets job-mode COOL + wind + power-save + temp + schedule, but omits airConOperationMode=POWER_ON.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.airConOperationMode?.ToLower()).AnyOf(["power_on"])

Failed

IN5 100 OUT170

×

SimpleUberEatPitadaBFCL/live_simple_27-7-0 (uber.eat.order): restaurant='uber pitada', items=[burgers, chicken wings], quantities=[5, 6].1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 712 OUT180

×

SimpleWeatherNaplesBFCL/live_simple_11-3-7 (get_current_weather): location='Naples, FL' or 'Naples, Florida'; unit fahrenheit or omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 426 OUT291

Tool Calling - Simple: Live Assistants, Weather & Devices (set 9) 3 failed

0.0%

IN152 339 OUT3 817

×

SimpleChafodWhopperBurgerBFCL/live_simple_22-5-0 (ChaFod): TheFod='BURGER' (Whopper mapped to burger).1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN58 235 OUT1 065

×

SimpleUberEatKfcChineseBFCL/live_simple_28-7-1 (uber.eat.order): KFC order, 4 items, quantities [10,50,30,90]. Prompt is Chinese, copied verbatim.1 AI query 19 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 19

Failed

IN81 368 OUT2 398

×

SimpleWeatherNewYorkBFCL/live_simple_12-3-8 (get_current_weather): location='New York, NY', unit=celsius.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 736 OUT354

Tool Calling - Simple: Live Media, Weather & Data (set 2) 3 passed7 failed

30.0%

IN147 897 OUT3 022

×

SimpleCreditQuoteAutoCreditoBFCL/live_simple_67-31-0 (obtener_cotizacion_de_creditos): monto=1000000, plazo=12, producto='auto', año=2024, enganche=0.2.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 765 OUT818

×

SimpleCurrentLocDmsBFCL/live_simple_87-48-0 (get_current_loc): coordinate_format='DMS', include_altitude=true, timeout default/omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 531 OUT173

×

SimpleGetMoviesMumbaiTodayBFCL/live_simple_59-28-0 (get_movies): city='Mumbai', optionals default/omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 840 OUT318

×

SimplePlayArtistKaarijaTracksBFCL/live_simple_93-54-0 (play_artist): artist_name='Käärijä'.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN33 436 OUT361

✓

SimpleRetrieveAnalysisSuperfancyBFCL/live_simple_69-33-0 (analysis_api.AnalysisApi.retrieve_analysis): project/component/vulnerability UUIDs.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 685 OUT218

×

SimpleSensorAlertsPerPage10BFCL/live_simple_70-34-0 (get_sensor_alerts): perPage=10, sensorSerial='Q3CC-CRT3-SZ2G', rest omitted.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN29 278 OUT626

✓

SimpleSitefinityNewsItemJulian12BFCL/live_simple_81-42-0 (sitefinity_create_contentitem): ContentItem='NewsItem', Title='Julian is testing12', rest omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 636 OUT130

✓

SimpleTodoAddShopping9pmB25BFCL/live_simple_55-25-1 (todo): type='add', content='go for shopping at 9 pm'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 579 OUT82

×

SimpleTodoDeleteTodoRandomBFCL/live_simple_54-25-0 (todo): type='delete', content='todo random'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.content?.ToLower()).AnyOf(["todo random"])

Failed

IN7 581 OUT66

×

SimpleWeatherGetLondonUkBFCL/live_simple_77-38-0 (weather.get): city='London', country='GB' (from UK), units/forecast default/omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 566 OUT230

Tool Calling - Simple: Live Media, Weather & Data (set 3) 6 passed4 failed

60.0%

IN120 614 OUT2 713

×

SimpleExtractMillennialsAppleBFCL/live_simple_71-35-0 (extract_parameters_v1): demographics=[millennials], targets=[brand:Apple], metrics=[view], min_date=2022-07-01. NOTE: targets @param clarified to '<attribute>:<entity>' order (was ambiguous 'target:target_type', which made the model emit the reversed 'Apple:brand'); GT 'brand:Apple' kept strict.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN37 157 OUT1 448

×

SimpleFetchPersonId4BFCL/live_simple_91-52-0 (fetchPerson): id=4.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 730 OUT331

×

SimpleLogFoodChaiTeaBFCL/live_simple_88-49-0 (log_food): food_name='chai tea', portion_amount=16.0, portion_unit='ounces', meal_name='snack'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 015 OUT162

✓

SimplePlaySongChaChaChaBFCL/live_simple_94-55-0 (play_song): artist_name='Käärijä', song_name='Cha Cha Cha'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 159 OUT79

✓

SimpleSendEmailAndyForecastBFCL/live_simple_78-39-0 (send_email): to='andy@gorilla.ai', subject='Sales Forecast Request', body set, cc/bcc/attachments omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 207 OUT172

✓

SimpleSitefinityNewsJulianTestingBFCL/live_simple_82-43-0 (sitefinity_create_contentitem): ContentItem='News', Title='Julian is testing', rest omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 175 OUT107

×

SimpleTemperatureParisFranceBFCL/live_simple_90-51-0 (temperature): location='Paris, France', units/time default/omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 722 OUT154

✓

SimpleTodoManagerAddSleepBFCL/live_simple_57-26-1 (todo_manager.handle_action): type='add', content='go to sleep at 9 pm'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 146 OUT77

✓

SimpleTodoManagerDeleteGymBFCL/live_simple_56-26-0 (todo_manager.handle_action): type='delete', content='go to gym'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 136 OUT71

✓

SimpleTranslateEnglishFrenchBFCL/live_simple_76-37-0 (language_translator.translate): source='en', target='fr', text='What is your name?'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 167 OUT112

Tool Calling - Simple: Live Media, Weather & Data (set 4) 8 passed2 failed

80.0%

IN65 212 OUT1 566

✓

SimpleOpenMeteoBerkeleyBFCL/live_simple_85-46-0 (open_meteo_api.fetch_weather_data): coordinates=[37.8715,-122.273], units default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 348 OUT108

×

SimpleSearchHotelsSanDiegoBFCL/live_simple_79-40-0 (search_hotels): location='San Diego, CA'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN8 270 OUT422

✓

SimpleSitefinityNewsSleepProductivityBFCL/live_simple_83-44-0 (sitefinity_create_contentitem): full news article with all metadata.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 482 OUT227

✓

SimpleSortArrayAscendingBFCL/live_simple_86-47-0 (sort_array): array=[1,54,3,1,76,2343,21], reverse default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 364 OUT111

×

SimpleTodoAddGoToGoaBFCL/live_simple_64-29-4 (todo): type='add', content='go to Goa'.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN14 273 OUT305

✓

SimpleTodoAddGymTomorrowBFCL/live_simple_61-29-1 (todo): type='add', content='go to gym tomorrow.'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 284 OUT72

✓

SimpleTodoAddHiCharlieBFCL/live_simple_65-29-5 (todo): type='add', content='Hi charlie'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 295 OUT69

✓

SimpleTodoAddSleep9pmBFCL/live_simple_63-29-3 (todo): type='add', content='go to sleep at 9 pm'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 291 OUT113

✓

SimpleTodoCompleteFinancialReportBFCL/live_simple_60-29-0 (todo): type='complete', content='Submit monthly financial report'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 308 OUT70

✓

SimpleTodoDeleteRaviBFCL/live_simple_62-29-2 (todo): type='delete', content='ravi'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 297 OUT69

Tool Calling - Simple: Live Movies, Music & Booking (set 1) 2 passed1 failed

66.7%

IN36 547 OUT832

×

SimpleMoviesTarantinoWhitakerBFCL/live_simple_208-117-0 (Movies_3_FindMovies): directed_by='Quentin Tarantino', cast='Duane Whitaker', genre dontcare.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN26 828 OUT690

✓

SimplePlaySongRoarBFCL/live_simple_230-121-0 (play_song): song_id='wjeiruhuq_roar', artist='Katy Perry', quality='high', device_id omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 887 OUT86

✓

SimpleSetVolume20BFCL/live_simple_242-126-0 (set_volume): volume=20.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 832 OUT56

Tool Calling - Simple: Live Movies, Music & Booking (set 10) 3 failed

0.0%

IN192 102 OUT4 536

×

SimpleHandoverPackageArrivalBFCL/live_simple_257-137-1 (answer_question): statement=package-arrival-time question, urgency/language default. System instructs HANDOVER on question. NOTE: no collider loop observed (single clean call, sibling tests pass); statement is free text -> substring-intent of the key phrase (model drops "Can you tell me").1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN25 328 OUT805

×

SimpleMoviesNiteshDirectorBFCL/live_simple_217-117-9 (Movies_3_FindMovies): directed_by='Nitesh Tiwari', genre/cast dontcare.1 AI query 29 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 29

Failed

IN158 747 OUT3 443

×

SimpleTtsChineseMaleBFCL/live_simple_227-118-1 (text_to_speech.convert): text=Chinese phrase, language='zh-CN', gender='male', format='wav', speed default.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN8 027 OUT288

Tool Calling - Simple: Live Movies, Music & Booking (set 11) 3 failed

0.0%

IN287 617 OUT4 050

×

SimpleMoviesColaizzoWatkinsBFCL/live_simple_218-117-10 (Movies_3_FindMovies): directed_by='Paul Downs Colaizzo', cast='Michaela Watkins', genre dontcare.1 AI query 43 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 43

Failed

IN269 580 OUT3 559

×

SimpleRequestsGetBuProductsBFCL/live_simple_229-120-0 (requests.get): anchor='user'.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 376 OUT322

×

SimpleTtsPrettyGirlDefaultBFCL/live_simple_228-119-0 (text_to_speech.convert): text='I am a pretty girl', language/gender/speed default.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 661 OUT169

Tool Calling - Simple: Live Movies, Music & Booking (set 12) 1 passed2 failed

33.3%

IN72 990 OUT2 252

×

SimpleBookFlightParisBFCL/live_simple_233-123-0 (book_flight): departure_location='Paris, France', departure_time='12/03/2023 15:00', return_time omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 422 OUT293

✓

SimpleGetVersionBFCL/live_simple_247-129-0 (version_api.VersionApi.get_version): no arguments.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 839 OUT74

×

SimpleMoviesSimonCurtisDramaBFCL/live_simple_219-117-11 (Movies_3_FindMovies): directed_by='Simon Curtis', genre='Drama', cast='McKinley Belcher III'.1 AI query 15 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN57 729 OUT1 885

Tool Calling - Simple: Live Movies, Music & Booking (set 13) 1 passed2 failed

33.3%

IN84 091 OUT3 605

✓

SimpleAclRetrieveProjectsBFCL/live_simple_248-130-0 (acl_api.AclApi.retrieve_projects): uuid='12', excludeInactive=true, onlyRoot=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 058 OUT1 817

×

SimpleBookFlightNewYorkToParisBFCL/live_simple_234-123-1 (book_flight): departure_location='New York, NY', departure_time='20/06/2022 17:00', return_time omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 786 OUT266

×

SimpleMoviesMackendrickThrillerBFCL/live_simple_220-117-12 (Movies_3_FindMovies): directed_by='Alexander Mackendrick', genre='Thriller', cast dontcare.1 AI query 17 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 17

Failed

IN68 247 OUT1 522

Tool Calling - Simple: Live Movies, Music & Booking (set 14) 3 failed

0.0%

IN96 829 OUT2 537

×

SimpleBookFlightJfkBFCL/live_simple_245-127-0 (book_flight): departure_location='JFK', departure_date='16/03/2024', departure_time='14:00'.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 590 OUT277

×

SimpleChatWithBunnyBFCL/live_simple_249-131-0 (chat_with_friend): user_message='Hey bunny, How are you doing'. NOTE: no collider loop observed (single clean call, sibling tests pass); user_message is a free-text passthrough -> substring-intent (model may drop the "Hey bunny," greeting).1 AI query 14 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 14

Failed

IN51 581 OUT1 227

×

SimpleMoviesJamesGrayBFCL/live_simple_221-117-13 (Movies_3_FindMovies): directed_by='James Gray', genre/cast dontcare.1 AI query 10 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN34 658 OUT1 033

Tool Calling - Simple: Live Movies, Music & Booking (set 15) 1 passed1 failed

50.0%

IN27 802 OUT830

✓

SimpleBookFlightNewYorkMarchBFCL/live_simple_246-128-0 (book_flight): departure_location='New York, NY', departure_time='15/03/2024'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 646 OUT137

×

SimpleMoviesVanOrmanBrownBFCL/live_simple_222-117-14 (Movies_3_FindMovies): directed_by='Thurop Van Orman', cast='Sterling K. Brown', genre dontcare.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN23 156 OUT693

Tool Calling - Simple: Live Movies, Music & Booking (set 16) 1 failed1 skipped

0.0%

IN293 660 OUT3 804

×

SimpleMoviesImaginativeFantasyBFCL/live_simple_223-117-15 (Movies_3_FindMovies): genre='Fantasy' ('Imaginative fiction'), director/cast dontcare.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 50

Failed

IN293 660 OUT3 804

–

SimpleRescheduleGymSessionBFCL/live_simple_231-122-0 (reschedule_event): event_identifier='456123', new_datetime='2022-10-30T16:30:00Z'.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: source flaw: relative-date arithmetic is inconsistent — prompt states 'Monday 2022-10-20' (that date is actually a Thursday) and the GT 'next Thursday' resolves to 2022-10-30 (a Sunday), so the exact new_datetime cannot be derived deterministically.

Skipped

IN0 OUT0

Tool Calling - Simple: Live Movies, Music & Booking (set 17) 1 passed1 failed

50.0%

IN20 184 OUT716

×

SimpleMoviesTimBurtonDeppBFCL/live_simple_224-117-16 (Movies_3_FindMovies): directed_by='Tim Burton', genre='Offbeat', cast='Johnny Depp'.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN15 503 OUT599

✓

SimpleRescheduleMedAppointmentBFCL/live_simple_232-122-1 (reschedule_event): event_identifier='med123', new_datetime='2021-04-15T13:45:00Z'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 681 OUT117

Tool Calling - Simple: Live Movies, Music & Booking (set 18) 1 passed1 failed

50.0%

IN19 658 OUT587

×

SimpleMoviesKirillComedyDramaBFCL/live_simple_225-117-17 (Movies_3_FindMovies): directed_by='Kirill Mikhanovsky', genre='Comedy-drama', cast dontcare.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN15 212 OUT531

✓

SimpleSetVolume70BFCL/live_simple_244-126-2 (set_volume): volume=70.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 446 OUT56

Tool Calling - Simple: Live Movies, Music & Booking (set 2) 2 passed1 failed

66.7%

IN20 129 OUT399

×

SimpleMoviesLeitchTuisanoBFCL/live_simple_209-117-1 (Movies_3_FindMovies): directed_by='David Leitch', cast='Lori Pelenise Tuisano', genre dontcare.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 390 OUT274

✓

SimpleSetVolume30BFCL/live_simple_243-126-1 (set_volume): volume=30.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 868 OUT58

✓

SimpleSpotifyFriendsMarshmelloBFCL/live_simple_235-124-0 (play_spotify_song): query='track:Friends artist:Marshmello', shuffle/volume default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 871 OUT67

Tool Calling - Simple: Live Movies, Music & Booking (set 3) 2 passed1 failed

66.7%

IN46 289 OUT956

×

SimpleFlightStatusIndigo6E123BFCL/live_simple_251-133-0 (flight.status.check): flight_id='6E123', passenger_name='Karan', ticket_number='IND4567'.1 AI query 10 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN36 176 OUT819

✓

SimpleMoviesDramaBFCL/live_simple_210-117-2 (Movies_3_FindMovies): genre='Drama', director/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 056 OUT63

✓

SimpleSpotifyDilNu124BFCL/live_simple_236-124-1 (play_spotify_song): query='track:Dil Nu artist:Maninder Buttar', shuffle/volume default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 057 OUT74

Tool Calling - Simple: Live Movies, Music & Booking (set 4) 3 failed

0.0%

IN131 893 OUT2 645

×

SimpleFlightStatusIndigoAirlineBFCL/live_simple_252-134-0 (flight.status.check): airlines='Indigo', person_name='Karan', ticket_number='IND4567'. NOTE: tool doc says "if only one name mentioned ... use LNU", so "Karan LNU" is equivalent-correct per the documented convention -> accept both.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN36 044 OUT796

×

SimpleMoviesComedyCordenBFCL/live_simple_211-117-3 (Movies_3_FindMovies): genre='Comedy' (prompt 'Comic'), cast='James Corden', director dontcare. NOTE: args are correct; single-call contract stays strict. The 27B occasionally emits a 2nd call (over-call) -> legit intermittent fail (kept as signal).1 AI query 21 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 21

Failed

IN90 813 OUT1 780

×

SimpleSpotifyWreckingBallBFCL/live_simple_237-125-0 (play_spotify_song): query='track:Wrecking Ball artist:Miley Cyrus', device_id/play default. NOTE: GT accepts only the structured 'track:.. artist:..' form (kept strict). The 27B occasionally drops the format (e.g. "wrecking ball Miley Cyrus") -> legit intermittent miss (kept as signal).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.query?.ToLower()).AnyOf(["track:wrecking ball artist:miley cyrus"])

Failed

IN5 036 OUT69

Tool Calling - Simple: Live Movies, Music & Booking (set 5) 1 passed2 failed

33.3%

IN85 455 OUT1 638

×

SimpleMoviesComedyEdgarWrightBFCL/live_simple_212-117-4 (Movies_3_FindMovies): directed_by='Edgar Wright', genre='Comedy', cast dontcare.1 AI query 17 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 17

Failed

IN66 608 OUT1 237

✓

SimpleSpotifyDilNu125BFCL/live_simple_238-125-1 (play_spotify_song): query='track:Dil Nu artist:Maninder Buttar', device_id/play default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 041 OUT80

×

SimpleUberEatsBigMacBFCL/live_simple_250-132-0 (uber.eat.order): restaurants='McDonald's', items=[Big Mac, McFlurry], quantities=[2,1].1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 806 OUT321

Tool Calling - Simple: Live Movies, Music & Booking (set 6) 1 passed2 failed

33.3%

IN35 743 OUT1 176

×

SimpleMoviesOffbeatTimBurtonBFCL/live_simple_213-117-5 (Movies_3_FindMovies): directed_by='Tim Burton', genre='Offbeat', cast dontcare.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 601 OUT643

✓

SimpleSpotifyShapeOfYouBFCL/live_simple_239-125-2 (play_spotify_song): query='track:Shape of You artist:Ed Sheeran' or 'Shape of You'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 040 OUT71

×

SimpleUberEatsBurgersWingsBFCL/live_simple_253-135-0 (uber.eat.order): restaurants='McDonald's', items=[burgers, chicken wings], quantities=[5,6].1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN17 102 OUT462

Tool Calling - Simple: Live Movies, Music & Booking (set 7) 1 passed2 failed

33.3%

IN83 931 OUT1 853

×

SimpleMoviesComicNiteshTiwariBFCL/live_simple_214-117-6 (Movies_3_FindMovies): directed_by='Nitesh Tiwari', genre='Comedy' or 'Comedy-drama', cast dontcare.1 AI query 17 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 17

Failed

IN68 516 OUT1 508

✓

SimpleSpotifyBabySharkBFCL/live_simple_240-125-3 (play_spotify_song): query='Baby Shark' (artist unknown).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 953 OUT63

×

SimpleUserMandatesPendingBFCL/live_simple_254-136-0 (user.mandates): user_id='U123', status='pending'.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 462 OUT282

Tool Calling - Simple: Live Movies, Music & Booking (set 8) 2 passed1 failed

66.7%

IN144 559 OUT2 511

×

SimpleMoviesFantasyTonightBFCL/live_simple_215-117-7 (Movies_3_FindMovies): genre='Fantasy', director/cast dontcare.1 AI query 28 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 28

Failed

IN134 621 OUT2 159

✓

SimpleSpotifyJohnnyPapaBFCL/live_simple_241-125-4 (play_spotify_song): query='Johnny Johnny Yes Papa' (artist unknown).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 949 OUT65

✓

SimpleUserMandatesActiveBFCL/live_simple_255-136-1 (user.mandates): user_id='U123456', status='active' or omitted (default).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 989 OUT287

Tool Calling - Simple: Live Movies, Music & Booking (set 9) 3 failed

0.0%

IN254 857 OUT4 703

×

SimpleHandoverComplaintStatusBFCL/live_simple_256-137-0 (answer_question): statement=user complaint-status question, urgency/language default. System instructs HANDOVER on question.1 AI query 38 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 38

Failed

IN222 519 OUT3 111

×

SimpleMoviesActionLeitchKingBFCL/live_simple_216-117-8 (Movies_3_FindMovies): directed_by='David Leitch', genre='Action', cast='Alex King'.1 AI query 7 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN24 362 OUT1 313

×

SimpleTtsPrettyGirlSpanishBFCL/live_simple_226-118-0 (text_to_speech.convert): text='I am a pretty girl' or Spanish translation, language='es-ES', gender/format/speed default.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 976 OUT279

Tool Calling - Simple: Live Services, Weather & Commands (set 1) 4 failed

0.0%

IN73 690 OUT1 532

×

SimpleExtractChesterJaneBFCL/live_simple_189-114-0 (extractor.extract_information): [Chester,42],[Jane,43] (Jane a year older).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 387 OUT207

×

SimpleServiceIdIroningUnitIntBFCL/live_simple_174-100-0 (get_service_id): ironing service -> service_id=2, unit omitted/1. Thai prompt.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN40 581 OUT564

×

SimpleWeatherNewYorkBFCL/live_simple_193-116-1 (Weather_1_GetWeather): city=New York, date=2023-03-08.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 856 OUT401

×

SimpleWeatherPacificaBFCL/live_simple_192-116-0 (Weather_1_GetWeather): city=Pacifica, date=2023-04-11.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 866 OUT360

Tool Calling - Simple: Live Services, Weather & Commands (set 10) 3 passed6 failed

33.3%

IN95 442 OUT3 885

×

SimpleCmdDockerPsBFCL/live_simple_172-99-6 (cmd_controller.execute): is Docker running -> 'docker ps'.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN9 629 OUT213

×

SimpleCmdDockerStartBFCL/live_simple_168-99-2 (cmd_controller.execute): start Docker -> 'docker start'.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN9 622 OUT253

×

SimpleCmdEchoDateBFCL/live_simple_171-99-5 (cmd_controller.execute): today's date -> 'echo %date%'.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 044 OUT228

✓

SimpleCmdEchoTimeBFCL/live_simple_166-99-0 (cmd_controller.execute): current time -> 'echo %time%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 598 OUT145

×

SimpleCmdListCDriveBFCL/live_simple_170-99-4 (cmd_controller.execute): list C drive -> 'dir C:\'.1 AI query 12 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN40 422 OUT1 075

×

SimpleCmdRunPythonScriptBFCL/live_simple_169-99-3 (cmd_controller.execute): run python script with venv interpreter.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.command?.ToLower()).AnyOf(["d:/playground/pc_contoller/env/scripts/python.exe d:/playground/pc_contoller/test.py"])

Failed

IN4 662 OUT157

✓

SimpleCmdStartCalcBFCL/live_simple_173-99-7 (cmd_controller.execute): open calculator -> 'start calc'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 579 OUT55

✓

SimpleCmdTasklistCodeBFCL/live_simple_167-99-1 (cmd_controller.execute): is VS Code running -> tasklist filter.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 610 OUT1 238

×

SimpleProvidersChonburiDateRangeBFCL/live_simple_186-111-0 (get_service_providers): ironing, Chonburi, rating>=4.5, 12/03/2024 16:00-18:00.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 276 OUT521

Tool Calling - Simple: Live Services, Weather & Commands (set 11) 2 passed2 failed

50.0%

IN37 971 OUT1 827

✓

SimpleItemsCircleLogosBFCL/live_simple_162-96-1 (get_items): 4 circle-logo copyright items.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 830 OUT220

×

SimpleItemsLegoMovieBFCL/live_simple_163-96-2 (get_items): 7 Lego-movie copyright items.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 637 OUT320

✓

SimpleItemsSuperheroLogosBFCL/live_simple_161-96-0 (get_items): 12 superhero-logo copyright items.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 061 OUT291

×

SimpleProvidersChonburiNoLateBFCL/live_simple_187-112-0 (get_service_providers): ironing, Chonburi, rating>=4.5, 12/03/2024, never late. NOTE: the prompt gives a time range "16:00 - 18:00", so end_available_date is expected; the 27B occasionally omits it (~1/3 of runs) -> a legit model-reliability miss, not an assertion flaw.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN20 443 OUT996

Tool Calling - Simple: Live Services, Weather & Commands (set 12) 2 failed

0.0%

IN75 842 OUT1 820

×

SimpleCopyrightCocaColaBFCL/live_simple_164-97-0 (get_copyright_info): Coca-Cola logo, The Coca-Cola Company, confidence 100.0.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN34 812 OUT838

×

SimpleProfessionalCleaningRating2BFCL/live_simple_183-108-0 (getDataForProfessional): cleaning in Bangkok, rating>=2.0 -> avg_rating=2.0, province_id=1, service_id=1.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN41 030 OUT982

Tool Calling - Simple: Live Services, Weather & Commands (set 13) 3 failed

0.0%

IN58 904 OUT1 757

×

SimpleMpnSpecs3pak7BFCL/live_simple_191-115-1 (raptor.mpn.specs): identifier=3pak7, MPN, include_images omitted/false.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN9 667 OUT338

×

SimpleMpnSpecsAchdImagesBFCL/live_simple_190-115-0 (raptor.mpn.specs): identifier=ACHD, MPN, include_images=true.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 496 OUT527

×

SimpleProfessionalCleaningRating4BFCL/live_simple_184-109-0 (getDataForProfessional): cleaning in Bangkok, rating>4 -> service_id=1, province_id=1, rating=4.0.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN36 741 OUT892

Tool Calling - Simple: Live Services, Weather & Commands (set 14) 2 failed

0.0%

IN43 524 OUT1 437

×

SimpleExtractChineseNamesBFCL/live_simple_165-98-0 (extractor.extract_information): Chinese text -> [李雷,18],[李丽,21] (sister 3 yrs older).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN19 852 OUT528

×

SimpleProfessionalIroningChiangMaiBFCL/live_simple_188-113-0 (getDataForProfessional): ironing in Chiang Mai, Sankampang/Ton Pao, rating 4.5.1 AI query 7 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN23 672 OUT909

Tool Calling - Simple: Live Services, Weather & Commands (set 2) 3 failed

0.0%

IN166 471 OUT2 802

×

SimpleServiceIdIroningUnitSessionBFCL/live_simple_175-101-0 (get_service_id): ironing -> service_id=2, unit omitted/session. Thai prompt.1 AI query 31 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 31

Failed

IN142 263 OUT2 027

×

SimpleWeatherMartinezBFCL/live_simple_194-116-2 (Weather_1_GetWeather): city=Martinez, date=2023-04-25.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN9 326 OUT272

×

SimpleWeatherPaloAltoBFCL/live_simple_195-116-3 (Weather_1_GetWeather): city=Palo Alto, date=2023-04-25.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN14 882 OUT503

Tool Calling - Simple: Live Services, Weather & Commands (set 3) 3 failed

0.0%

IN160 054 OUT2 861

×

SimpleServiceIdIroningUnitQtyBFCL/live_simple_176-102-0 (get_service_id): ironing -> service_id=2, unit omitted/1. Thai prompt.1 AI query 30 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 30

Failed

IN135 900 OUT1 989

×

SimpleWeatherAlamedaWednesdayBFCL/live_simple_196-116-4 (Weather_1_GetWeather): city=Alameda, upcoming Wednesday (today Mon 2023-04-25) -> 2023-04-27.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN9 290 OUT415

×

SimpleWeatherStinsonBeachBFCL/live_simple_197-116-5 (Weather_1_GetWeather): city=Stinson Beach, date=2023-04-05.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN14 864 OUT457

Tool Calling - Simple: Live Services, Weather & Commands (set 4) 4 failed

0.0%

IN165 799 OUT2 612

×

SimpleServiceIdCleaningBFCL/live_simple_177-103-0 (get_service_id): cleaning -> service_id=1, unit omitted/1.1 AI query 27 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 27

Failed

IN112 428 OUT1 323

×

SimpleServiceIdIroningBFCL/live_simple_178-103-1 (get_service_id): ironing -> service_id=2, unit omitted/1.1 AI query 11 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN34 591 OUT743

×

SimpleWeatherHealdsburgBFCL/live_simple_198-116-6 (Weather_1_GetWeather): city=Healdsburg, date=2023-03-02.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 031 OUT343

×

SimpleWeatherMarshallBFCL/live_simple_199-116-7 (Weather_1_GetWeather): city=Marshall (prompt 'Marshall, MN'), date=2023-03-05. NOTE: GT city is bare "Marshall"; model faithfully copies the "Marshall, MN" disambiguator from the prompt -> substring-intent (equivalent-correct).1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN6 749 OUT203

Tool Calling - Simple: Live Services, Weather & Commands (set 5) 1 passed2 failed

33.3%

IN28 620 OUT875

✓

SimpleServiceIdIroningChiangMaiBFCL/live_simple_179-104-0 (get_service_id): ironing in Chiang Mai -> service_id=2, province_id=2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 378 OUT81

×

SimpleWeatherCampbellBFCL/live_simple_201-116-9 (Weather_1_GetWeather): city=Campbell, date=2023-03-04.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 151 OUT400

×

SimpleWeatherFremontBFCL/live_simple_200-116-8 (Weather_1_GetWeather): city=Fremont, date=2023-03-01.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 091 OUT394

Tool Calling - Simple: Live Services, Weather & Commands (set 6) 3 failed

0.0%

IN133 718 OUT2 549

×

SimpleServiceIdIroningBangkokRatingBFCL/live_simple_180-105-0 (get_service_id): ironing in Bangkok, rating>=4 -> service_id=2, province_id=1, rating=4.0.1 AI query 26 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 26

Failed

IN114 578 OUT1 885

×

SimpleWeatherFosterCityBFCL/live_simple_202-116-10 (Weather_1_GetWeather): city=Foster City, date=2023-04-25.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 241 OUT407

×

SimpleWeatherWashingtonBFCL/live_simple_203-116-11 (Weather_1_GetWeather): city=Washington (prompt 'Washington, DC'), date=2023-03-01. NOTE: GT city is bare "Washington"; model faithfully copies the "Washington, DC" disambiguator from the prompt -> substring-intent (equivalent-correct).1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN6 899 OUT257

Tool Calling - Simple: Live Services, Weather & Commands (set 7) 3 failed

0.0%

IN219 125 OUT3 543

×

SimpleServiceIdIroningChatuchakBFCL/live_simple_181-106-0 (get_service_id): ironing, Bangkok(Chatuchak), rating>=4.5 -> service_id=2, province_id omitted/1, district_name=Chatuchak, rating=4.5.1 AI query 36 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 36

Failed

IN196 935 OUT2 818

×

SimpleWeatherBerkeleySaturdayBFCL/live_simple_205-116-13 (Weather_1_GetWeather): city=Berkeley, upcoming Saturday (today Tue 2023-04-25) -> 2023-04-29.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN9 684 OUT402

×

SimpleWeatherRutherfordBFCL/live_simple_204-116-12 (Weather_1_GetWeather): city=Rutherford (prompt 'Rutherford, NJ'), date=2023-04-22. NOTE: GT city is bare "Rutherford"; model faithfully copies the "Rutherford, NJ" disambiguator from the prompt -> substring-intent (equivalent-correct).1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 506 OUT323

Tool Calling - Simple: Live Services, Weather & Commands (set 8) 3 failed

0.0%

IN144 957 OUT3 433

×

SimpleProvidersKhlongSamWaBFCL/live_simple_182-107-0 (get_service_providers): ironing, Bangkok, Khlong Sam Wa / Sai Kong, rating>=4.5.1 AI query 26 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 26

Failed

IN125 324 OUT2 797

×

SimpleWeatherLondonBFCL/live_simple_206-116-14 (Weather_1_GetWeather): city=London (prompt 'London, England'), date=2023-03-05.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 551 OUT358

×

SimpleWeatherSacramentoBFCL/live_simple_207-116-15 (Weather_1_GetWeather): city=Sacramento, date=2023-04-22.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 082 OUT278

Tool Calling - Simple: Live Services, Weather & Commands (set 9) 3 passed3 failed

50.0%

IN125 618 OUT3 149

✓

SimpleCmdEchoTimeUnitBFCL/live_simple_157-95-14 (cmd_controller.execute): current time -> 'echo %time%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 623 OUT163

×

SimpleCmdListCDriveUnitBFCL/live_simple_156-95-13 (cmd_controller.execute): list C drive -> 'dir C:\'.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN22 304 OUT306

✓

SimpleCmdStartCameraBFCL/live_simple_160-95-17 (cmd_controller.execute): open camera -> 'start microsoft.windows.camera:'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 618 OUT57

×

SimpleCmdTaskkillCodeBFCL/live_simple_158-95-15 (cmd_controller.execute): kill VS Code -> 'taskkill /IM code.exe /F'.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN36 519 OUT788

✓

SimpleCmdTimeout10BFCL/live_simple_159-95-16 (cmd_controller.execute): timer 10s -> 'timeout 10'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 638 OUT84

×

SimpleProvidersChonburiTomorrowBFCL/live_simple_185-110-0 (get_service_providers): ironing, Chonburi, rating>=4.5, tomorrow of 20230929 -> 2023-09-30.1 AI query 14 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 14

Failed

IN52 916 OUT1 751

Tool Calling - Simple: Live Shell, Classification & APIs (set 1) 2 passed1 failed

66.7%

IN35 049 OUT1 754

✓

SimpleClassifyGreetingsAndLinksBFCL/live_simple_105-62-0 (classify): clean_hello=['hello'], faq_link_accounts_start=['link my account','connect my accounts'].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 601 OUT94

×

SimpleCmdDockerVersionBFCL/live_simple_141-94-0 (cmd_controller.execute): command='docker --version'.1 AI query 8 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN25 742 OUT1 455

✓

SimpleReqNodesPekAciBFCL/live_simple_128-83-0 (requests.get): nodes list url with ip 192.168.11.33, fabricName='PEK-ACI'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 706 OUT205

Tool Calling - Simple: Live Shell, Classification & APIs (set 10) 2 passed2 failed

50.0%

IN33 697 OUT941

✓

SimpleCmdShutdownBFCL/live_simple_150-95-7 (cmd_controller.execute): shutdown the pc via 'shutdown /s /t 0'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 870 OUT77

×

SimpleProcessImageMoondreamBFCL/live_simple_122-78-0 (process_data): image.png + long question + model 'vikhyatk/moondream2'.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 631 OUT524

×

SimpleReqDataLimit50BFCL/live_simple_135-88-0 (requests.get): data endpoint url, params limit=50.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 324 OUT267

✓

SimpleSumNumbersListBFCL/live_simple_116-72-0 (sum_numbers): numbers_list=[133, 34].1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 872 OUT73

Tool Calling - Simple: Live Shell, Classification & APIs (set 11) 3 passed1 failed

75.0%

IN28 815 OUT1 327

✓

SimpleCmdWifiDisconnectBFCL/live_simple_151-95-8 (cmd_controller.execute): disconnect wifi via 'netsh wlan disconnect'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 897 OUT66

✓

SimplePipelineFurnitureImageBFCL/live_simple_123-79-0 (pipeline): analyze thewowstyle image with furniture/minimalist question.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 173 OUT543

×

SimpleReqTopologySalesAppBFCL/live_simple_136-89-0 (requests.get): topology url ip 192.168.1.1, filter accountName:AcmeCorp AND applicationName:SalesApp.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 830 OUT395

✓

SimpleTemperatureHyderabadBFCL/live_simple_118-74-0 (get_temperature): units=Celsius, location='Hyderabad, India', time omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 915 OUT323

Tool Calling - Simple: Live Shell, Classification & APIs (set 12) 1 passed3 failed

25.0%

IN39 188 OUT1 429

×

SimpleCmdDirDAndEchoBFCL/live_simple_152-95-9 (cmd_controller.execute): 'dir D:\ && echo testing.txt'.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN11 034 OUT258

×

SimpleReqAnomaliesSummaryBFCL/live_simple_139-92-0 (requests.get): anomalies summary url ip 192.168.1.1, params fabric='network222'.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN11 462 OUT531

✓

SimpleRescheduleAlice8pmBFCL/live_simple_137-90-0 (reschedule): identifier='Alice-One-one-One', 2023-11-01T20:00:00, Europe/London.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 310 OUT119

×

SimpleWeatherForecastParisBFCL/live_simple_120-76-0 (weather.forecast): Paris, France 2023-04-03..05, Celsius, defaults omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN11 382 OUT521

Tool Calling - Simple: Live Shell, Classification & APIs (set 13) 2 passed1 failed

66.7%

IN59 508 OUT1 667

✓

SimpleCmdDelPuttyBFCL/live_simple_153-95-10 (cmd_controller.execute): remove putty shortcut via del with quoted path.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 822 OUT69

×

SimpleDynamicPricingSaaSBFCL/live_simple_113-69-0 (calculate_dynamic_pricing): geolocation='34.0522, -118.2437', base=100, min=90.1 AI query 13 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 13

Failed

IN49 828 OUT1 475

✓

SimpleRescheduleBob4pmBFCL/live_simple_138-91-0 (reschedule): identifier='Bob-123', 2023-11-01T16:00:00, Europe/London.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 858 OUT123

Tool Calling - Simple: Live Shell, Classification & APIs (set 14) 2 passed1 failed

66.7%

IN100 916 OUT1 715

✓

SimpleCmdStartYoutubeBFCL/live_simple_154-95-11 (cmd_controller.execute): open youtube rickroll via start command.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 715 OUT75

×

SimpleGetTicketsCompanyOpenBFCL/live_simple_124-80-0 (get_tickets): customer (company name in prompt), status=open, limit=10.1 AI query 22 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 22

Failed

IN91 510 OUT1 456

✓

SimpleGreetPartnerJeffBFCL/live_simple_140-93-0 (greet_partner): user='Emily', partner='Jeff', relationship='husband'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 691 OUT184

Tool Calling - Simple: Live Shell, Classification & APIs (set 15) 3 failed

0.0%

IN34 889 OUT1 574

×

SimpleCalculateTaxNewYorkBFCL/live_simple_104-61-2 (calculate_tax): purchase_amount=250, state='NY', all optionals omitted/default.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 427 OUT820

×

SimpleCmdDateBFCL/live_simple_155-95-12 (cmd_controller.execute): what is the date via 'date /t' or 'echo %date%'.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN22 628 OUT689

×

SimpleConcatJohnDoeBFCL/live_simple_125-81-0 (concat_strings): strings=['John','Doe'], separator=' '. EXPECTED MODEL MISS: 'combine ... into a full name' requires separator=' ' (John Doe). The 27B tends to omit separator (=> 'JohnDoe'); that is a legit model failure of the measured competence, so the ' ' assertion stays strict.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.separator.AnyOf([" "]) Values: args.separator = null

Failed

IN4 834 OUT65

Tool Calling - Simple: Live Shell, Classification & APIs (set 2) 1 passed2 failed

33.3%

IN40 028 OUT2 802

×

SimpleCmdDirDesktopBFCL/live_simple_142-94-1 (cmd_controller.execute): command='dir Desktop'.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN16 882 OUT634

✓

SimpleRecordBankNineIntentsBFCL/live_simple_106-63-0 (record): classify 20 bank queries into 9 intents (2 buckets empty).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 967 OUT827

×

SimpleReqNodesFabEdBFCL/live_simple_129-83-1 (requests.get): nodes list url with ip 3.321.3232.2, fabricName='fab-ed'.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN17 179 OUT1 341

Tool Calling - Simple: Live Shell, Classification & APIs (set 3) 2 passed2 failed

50.0%

IN36 219 OUT1 164

×

SimpleCmdDockerPsBFCL/live_simple_143-95-0 (cmd_controller.execute): command='docker ps'.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN14 008 OUT303

✓

SimpleRecordTelephoneHoursAccountsBFCL/live_simple_107-64-0 (record): classify 7 queries into telephone-banking / bank-hours / accounts.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 644 OUT348

×

SimpleTelemetryGlobalFabricStatusBFCL/live_simple_126-82-0 (telemetry.flowrules.interfaceInfo.get): fabric='Global-Fabric', node=1200, pod=3, ethernet, status.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN11 254 OUT330

✓

SimpleUpdateProfileEmailAgeBFCL/live_simple_114-70-0 (update_user_profile): user_id=12345, email+age set, bio omitted, notify default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 313 OUT183

Tool Calling - Simple: Live Shell, Classification & APIs (set 4) 2 passed2 failed

50.0%

IN29 834 OUT1 687

✓

SimpleCmdKillFirefoxBFCL/live_simple_144-95-1 (cmd_controller.execute): close firefox via taskkill.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 085 OUT66

×

SimpleRecordMoneyTransactionsCreditBFCL/live_simple_108-65-0 (record): classify 10 queries into money-movement / transactions / credit-limit / link-accounts. EXPECTED MODEL MISS: the 27B classifies 'show me all outgoing wire transfers' as a transaction-view (get_transactions_start) rather than money_movement_start per GT, so the get_transactions count comes out as 5 instead of 4. GT bucketing kept strict; this is a legit model classification failure.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN8 718 OUT1 196

✓

SimpleReverseSayHiBFCL/live_simple_117-73-0 (reverse_input): input_value='say hi'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 073 OUT70

×

SimpleTelemetryTestDeStatusBFCL/live_simple_127-82-1 (telemetry.flowrules.interfaceInfo.get): fabric='test-de', node=5, pod=1, ethernet, status.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 958 OUT355

Tool Calling - Simple: Live Shell, Classification & APIs (set 5) 2 passed2 failed

50.0%

IN49 155 OUT1 988

×

SimpleCmdDirCDriveBFCL/live_simple_145-95-2 (cmd_controller.execute): list C drive => 'dir C:\'. EXPECTED MODEL MISS: for the terse/typo'd 'LIST C DRIIVE' the 27B currently returns 'dir d:' — the wrong drive. The accepted-command list is broadened with the no-backslash 'dir c:' variant for fairness, but a wrong drive letter is a legit model failure.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN20 262 OUT578

✓

SimplePrimeMinistersThreeBFCL/live_simple_119-75-0 (GetPrimeMinisters): countries=[Australia, Canada, India].1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 143 OUT84

✓

SimpleRecordAutoWithdrawIncomeBFCL/live_simple_109-66-0 (record): classify 5 loan/withdraw queries; outofscope empty.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 361 OUT280

×

SimpleReqInterfaceStatsNetwork1BFCL/live_simple_130-84-0 (requests.get): interfaceInfo stats, ip 192.120.45.67, Network1/node 12/pod 10/ethernet/statistics.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN18 389 OUT1 046

Tool Calling - Simple: Live Shell, Classification & APIs (set 6) 2 passed2 failed

50.0%

IN39 904 OUT951

✓

SimpleCmdStartHuggingfaceBFCL/live_simple_146-95-3 (cmd_controller.execute): open huggingface via start command.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 296 OUT81

×

SimpleGenerateSalaryChartBFCL/live_simple_121-77-0 (generate_chart): 5 job labels and salaries in order.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN14 540 OUT317

✓

SimpleRecordAtmLocationOnlyBFCL/live_simple_110-67-0 (record): single query 'Need ATM location' -> atm_finder_start only.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 301 OUT108

×

SimpleReqInterfaceSpeedNetwork1BFCL/live_simple_131-84-1 (requests.get): interfaceInfo Speed, ip 192.120.45.67, Network1/node 12/pod 10/ethernet/Speed.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN14 767 OUT445

Tool Calling - Simple: Live Shell, Classification & APIs (set 7) 1 passed2 failed

33.3%

IN26 035 OUT1 051

✓

SimpleCmdKillTimerBFCL/live_simple_147-95-4 (cmd_controller.execute): remove timer.exe via taskkill.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 869 OUT71

×

SimpleRecordBalanceAndAtmBFCL/live_simple_111-67-1 (record): 10 queries -> 8 balance + 2 atm; other buckets empty.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN8 010 OUT381

×

SimpleReqDeviceConnectorVersionsBFCL/live_simple_132-85-0 (requests.get): device connector versions JSON url, no params.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 156 OUT599

Tool Calling - Simple: Live Shell, Classification & APIs (set 8) 1 passed2 failed

33.3%

IN76 241 OUT2 104

×

SimpleCmdWhatFileInCBFCL/live_simple_148-95-5 (cmd_controller.execute): 'what is the file in c' => 'dir C:\'. EXPECTED MODEL MISS: the 27B mis-selects the 'record' classifier (dropping the request into its 'outofscope' bucket) instead of cmd_controller_execute. record is a legitimate distractor here (not a looping universal collider), so this stays as a real tool-selection failure — no isolation needed.1 AI query 15 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN60 705 OUT1 570

✓

SimpleRecordWeatherOutOfScopeBFCL/live_simple_112-68-0 (record): single query 'what is the weather like' -> outofscope only.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 962 OUT168

×

SimpleReqRecommendationsDacBFCL/live_simple_133-86-0 (requests.get): recommendations url, params advisoryId='dac'.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 574 OUT366

Tool Calling - Simple: Live Shell, Classification & APIs (set 9) 3 passed

100.0%

IN13 883 OUT336

✓

SimpleCalculateSum133And34BFCL/live_simple_115-71-0 (calculate_sum): number1=133, number2=34.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 611 OUT74

✓

SimpleCmdEchoHiBFCL/live_simple_149-95-6 (cmd_controller.execute): say hi via echo command.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 585 OUT63

✓

SimpleReqInsightsGroupsBFCL/live_simple_134-87-0 (requests.get): insights groups url, params limit=20, sort='desc'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 687 OUT199

Tool Calling - Simple: Math, Physics & Geometry (set 1) 12 failed

0.0%

IN314 957 OUT7 163

×

SimpleAlgebraQuadraticRootsBFCL/simple_python_3 (algebra.quadratic_roots): a=1, b=-3, c=2.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 825 OUT388

×

SimpleAreaUnderCurveXSquaredBFCL/simple_python_13 (calculate_area_under_curve): function y=x^2, interval [1,3], method omitted/trapezoidal.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN48 240 OUT960

×

SimpleCircleCircumferenceInchesBFCL/simple_python_7 (calculate_circumference): radius=4, unit inches/in.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 684 OUT392

×

SimpleDerivativeCubicPolyBFCL/simple_python_14 (calculate_derivative): function 3x^2+2x-1, x_value omitted or 0.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 400 OUT367

×

SimpleElectricFieldFromChargeBFCL/simple_python_39 (calculate_electric_field): charge=2, distance=3, permitivity omitted/8.854e-12.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN28 123 OUT662

×

SimpleFinalVelocityFromDistanceBFCL/simple_python_30 (kinematics.final_velocity_from_distance): acceleration=4, distance=300, initial_velocity omitted/0.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 960 OUT557

×

SimpleFinalVelocityFromHeightBFCL/simple_python_25 (calculate_final_velocity): height=150, initial_velocity omitted/0, gravity omitted/9.81.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN33 021 OUT683

×

SimpleGeometryAreaCircleBFCL/simple_python_8 (geometry.area_circle): radius=10, units omitted or 'meters'.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 519 OUT309

×

SimpleGetPrimeFactors450BFCL/simple_python_17 (get_prime_factors): number=450, formatted omitted or true.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN28 046 OUT643

×

SimpleMagneticFieldLoopBFCL/simple_python_40 (calculate_magnetic_field): current=5, radius=4, permeability omitted/12.57e10.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 261 OUT1 377

×

SimpleMathGcd40And50BFCL/simple_python_19 (math.gcd): num1=40, num2=50.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN28 158 OUT443

×

SimpleTriangleAreaWithUnitBFCL/simple_python_0 (calculate_triangle_area): base=10, height=5, unit omitted or 'units'.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 720 OUT382

Tool Calling - Simple: Math, Physics & Geometry (set 2) 12 failed

0.0%

IN330 702 OUT6 557

×

SimpleAverageVelocityBFCL/simple_python_26 (calculate_velocity): distance=50, duration=2, unit omitted or 'km/h'.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN32 864 OUT510

×

SimpleCalculusDerivativeAtValueBFCL/simple_python_16 (calculus.derivative): function 2x^2, value=1, function_variable omitted or 'x'.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN28 077 OUT469

×

SimpleElectricFieldStrengthBFCL/simple_python_44 (calculate_electric_field_strength): charge=0.01, distance=4, medium omitted or 'vacuum'.1 AI query 11 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN52 610 OUT913

×

SimpleFinalVelocityConstAccelBFCL/simple_python_31 (calculate_final_velocity): initial_velocity=0, acceleration=9.8, time=5.1 AI query 10 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN47 957 OUT767

×

SimpleGeometryCalculateAreaCircleBFCL/simple_python_9 (geometry.calculate_area_circle): radius=5, unit omitted or 'units'.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 183 OUT187

×

SimpleGeometryCircumferenceBFCL/simple_python_12 (geometry.circumference): radius=3, units omitted or 'cm'.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 102 OUT269

×

SimpleIntegrateCubeSimpsonBFCL/simple_python_15 (integrate): function x^3, start_x=-2, end_x=3, method simpson.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 948 OUT645

×

SimpleMagneticFieldStrengthWireBFCL/simple_python_43 (calculate_magnetic_field_strength): current=20, distance=10, permeability omitted/12.57e-7.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 669 OUT662

×

SimpleMathHcfBFCL/simple_python_20 (math.hcf): number1=36, number2=24.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 316 OUT241

×

SimpleNumberAnalysisPrimeFactorsBFCL/simple_python_18 (number_analysis.prime_factors): number=123456.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN27 920 OUT951

×

SimpleRightTriangleAreaCmBFCL/simple_python_10 (calculate_area): base=6, height=10, unit omitted or 'cm'.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 260 OUT286

×

SimpleSolveQuadraticEquation265BFCL/simple_python_4 (solve_quadratic_equation): a=2, b=6, c=5.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN32 796 OUT657

Tool Calling - Simple: Math, Physics & Geometry (set 3) 9 failed

0.0%

IN263 703 OUT6 390

×

SimpleBoilingMeltingPointsBFCL/simple_python_47 (get_boiling_melting_points): substance='water', sea_level=5000.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN25 813 OUT683

×

SimpleDisplacementBFCL/simple_python_28 (calculate_displacement): initial_velocity=10, time=5, acceleration=9.8.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN30 268 OUT770

×

SimpleElectrostaticPotentialBFCL/simple_python_38 (calculate_electrostatic_potential): charge1=1e-9, charge2=2e-9, distance=0.05, constant omitted/8.99e9.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 008 OUT914

×

SimpleFinalSpeedFromHeightBFCL/simple_python_32 (calculate_final_speed): initial_velocity=0, height=100, gravity omitted or 9.8.1 AI query 8 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN34 298 OUT642

×

SimpleNumberTheoryGcdBFCL/simple_python_21 (number_theory.gcd): number1=36, number2=48.1 AI query 11 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN48 620 OUT683

×

SimplePrimeFactorizeDictionaryBFCL/simple_python_23 (prime_factorize): number=60, return_type='dictionary'.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN29 924 OUT558

×

SimpleRouteEstimateTimeStopsBFCL/simple_python_37 (route.estimate_time): SF -> LA, stops {Santa Barbara, Monterey} in either order.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN26 143 OUT794

×

SimpleSolveQuadraticAllRootsBFCL/simple_python_5 (solve_quadratic): a=3, b=-11, c=-4, root_type='all' (user asked for all roots).1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN44 448 OUT1 163

×

SimpleTriangleAreaNoUnitBFCL/simple_python_11 (calculate_triangle_area): base=10, height=5.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 181 OUT183

Tool Calling - Simple: Math, Physics & Geometry (set 4) 9 failed

0.0%

IN196 938 OUT5 387

×

SimpleCalculateDensityBFCL/simple_python_48 (calculate_density): mass=45, volume=15, unit omitted or 'kg/m^3'.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN25 519 OUT401

×

SimpleElectromagneticForceBFCL/simple_python_41 (electromagnetic_force): charge1=5, charge2=7, distance=3, medium_permittivity omitted/8.854e-12.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 578 OUT569

×

SimpleFinalSpeedFreeFallTimeBFCL/simple_python_29 (calculate_final_speed): initial_speed omitted/0, time=5, gravity omitted/-9.81.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN29 879 OUT755

×

SimpleMathFactorial5BFCL/simple_python_1 (math.factorial): number=5.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 052 OUT177

×

SimpleMathGcd12And15BFCL/simple_python_22 (math.gcd): num1=12, num2=15.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 121 OUT129

×

SimpleShortestDrivingDistanceBFCL/simple_python_36 (get_shortest_driving_distance): origin=New York City, destination=Washington D.C., unit omitted/km.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 527 OUT401

×

SimpleSolveQuadraticRoots253BFCL/simple_python_6 (solve_quadratic): a=2, b=5, c=3.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN25 617 OUT690

×

SimpleThermoEnergyVaporizationBFCL/simple_python_45 (thermo.calculate_energy): mass=100, phase_transition=vaporization, substance omitted/water.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 726 OUT550

×

SimpleTravelItineraryTokyoBFCL/simple_python_34 (travel_itinerary_generator): destination=Tokyo, days=7, daily_budget=100, exploration_type=nature.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN34 919 OUT1 715

Tool Calling - Simple: Math, Physics & Geometry (set 5) 8 failed

0.0%

IN182 968 OUT4 235

×

SimpleAbsolutePressureBFCL/simple_python_49 (calc_absolute_pressure): atm_pressure=1, gauge_pressure=2.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 329 OUT485

×

SimpleFinalTemperatureMixBFCL/simple_python_46 (calculate_final_temperature): mass1=20, temperature1=30, mass2=15, temperature2=60, shc omitted/4.2.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN25 344 OUT724

×

SimpleFinalVelocityAccelTimeBFCL/simple_python_27 (final_velocity): initial_velocity=10, acceleration=2, time=5.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 450 OUT338

×

SimpleGetDirectionsFastestBFCL/simple_python_33 (get_directions): start=Sydney, end=Melbourne, route_type omitted or 'fastest'.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN20 966 OUT465

×

SimpleMathGcd12And18BFCL/simple_python_24 (math.gcd): num1=12, num2=18.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 011 OUT240

×

SimpleMathHypotenuseBFCL/simple_python_2 (math.hypot): x=4, y=5, z omitted or 0.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 765 OUT401

×

SimpleResonantFrequencyLCBFCL/simple_python_42 (calculate_resonant_frequency): inductance=0.05 (50mH), capacitance=0.0001 (100uF), round_off omitted/2.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN38 256 OUT858

×

SimpleVeganRestaurantFindBFCL/simple_python_35 (vegan_restaurant.find_nearby): location='New York, NY', operating_hours=23 (11 PM).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN28 847 OUT724

Tool Calling - Simple: Science, Data & Biology (set 1) 1 passed12 failed

7.7%

IN436 460 OUT9 290

×

SimpleArraySortBFCL/simple_python_87 (array_sort): list=[5,3,4,1,2], order=ascending.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN17 006 OUT245

×

SimpleAverageTemperatureBFCL/simple_python_78 (average_temperature): location=Austin, days=3, temp_unit=Celsius.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN16 906 OUT232

×

SimpleBacteriaEvolutionRateBFCL/simple_python_74 (calculate_bacteria_evolution_rate): start=5000, dup_freq=1, duration=6, gen_time omitted or 20.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN36 476 OUT670

×

SimpleBmiKgCmBFCL/simple_python_84 (calculate_bmi): weight=85, height=180, unit omitted or 'metric'.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN26 339 OUT495

×

SimpleCreateHistogramBFCL/simple_python_79 (create_histogram): data=[85,90,88,92,86,89,91], bins=5.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN26 627 OUT716

×

SimpleCreatureFitnessBFCL/simple_python_72 (calculate_fitness): trait_values=[0.8,0.7], trait_contributions=[0.4,0.6].1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN26 552 OUT472

×

SimpleDbFetchStudentsBFCL/simple_python_89 (db_fetch_records): database=StudentDB, table=students, conditions={department:Science, school:Bluebird High School}, fetch_limit omitted or 0.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN31 656 OUT658

✓

SimpleEntropyChangeSubstanceBFCL/simple_python_50 (entropy_change.calculate): substance=ice, mass=1, init=0, final=100, pressure omitted or 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 269 OUT1 288

×

SimpleGenerateDnaSequenceBFCL/simple_python_71 (generate_DNA_sequence): length=100, preferences=[G,C] in either order.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN26 332 OUT619

×

SimpleGpsCoordinateDistanceBFCL/simple_python_83 (calculate_distance): coord1=[33.4484,-112.074], coord2=[34.0522,-118.2437], unit=miles.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN17 250 OUT978

×

SimpleImdbMoviesByActorBFCL/simple_python_92 (imdb.find_movies_by_actor): actor='Leonardo DiCaprio', year=2010, category omitted or 'all'.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN26 390 OUT603

×

SimplePopulationProjectionBFCL/simple_python_73 (population_projections): country=United States/USA, years=20, growth_rate omitted or 1.2.1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN87 603 OUT979

×

SimpleRestaurantFindNearbyBFCL/simple_python_77 (restaurant.find_nearby): location='Los Angeles, CA', dietary_preference=[Vegan].1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN89 054 OUT1 335

Tool Calling - Simple: Science, Data & Biology (set 2) 13 failed

0.0%

IN801 258 OUT12 452

×

SimpleBmiMetersBFCL/simple_python_88 (calculate_BMI): weight_kg=70, height_m=1.75.1 AI query 8 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN41 309 OUT534

×

SimpleClockAngleBFCL/simple_python_98 (calculate_clock_angle): hours=6, minutes=30, round_to omitted or 2.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN51 932 OUT1 019

×

SimpleDatabaseQueryConditionsBFCL/simple_python_96 (database.query): table=user, conditions=[{age,>,25},{job,=,engineer}].1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN53 897 OUT1 079

×

SimpleElephantPopulationBFCL/simple_python_75 (elephant_population_estimate): current=35000, growth_rate=0.015, years=5.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN31 385 OUT738

×

SimpleEntropyChangeIsothermalBFCL/simple_python_51 (calculate_entropy_change): initial_temp=300, final_temp=400, heat_capacity=5, isothermal omitted or true.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN31 455 OUT989

×

SimpleEvolutionPredictionDarwinBFCL/simple_python_76 (prediction.evolution): species=Homo Sapiens, years=50, model=Darwin.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN52 520 OUT995

×

SimpleFindRestaurantsThaiBFCL/simple_python_80 (find_restaurants): location=Manhattan variants, food_type=Thai, number=5, dietary_requirements=[vegan].1 AI query 50 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN417 926 OUT4 974

×

SimpleGeoDistanceCitiesBFCL/simple_python_85 (geo_distance.calculate): start='Boston, MA', end='Washington, D.C.', units omitted or 'miles'.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN31 263 OUT571

×

SimpleMathFactorialBFCL/simple_python_97 (math.factorial): number=5.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN12 377 OUT210

×

SimplePlotSineWaveBFCL/simple_python_99 (plot_sine_wave): start=0.0, end=6.2832, frequency=5, amplitude omitted or 1, phase_shift omitted or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN12 488 OUT223

×

SimpleTheaterMovieReleasesBFCL/simple_python_93 (get_theater_movie_releases): location=LA, timeframe=7, format=IMAX.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN26 261 OUT567

×

SimpleTriangleAreaMetersBFCL/simple_python_95 (calc_area_triangle): base=5, height=3.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN21 361 OUT267

×

SimpleUpdateUserInfoBFCL/simple_python_94 (update_user_info): user_id=43523, update_info={name:John Doe, email:johndoe@email.com}, database omitted or CustomerInfo. NOTE: single-call contract kept intentionally; the model very occasionally emits a spurious second call (args are correct ~3/3 sampled runs), a model-reliability signal rather than a test defect.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN17 084 OUT286

Tool Calling - Simple: Science, Data & Biology (set 3) 11 failed1 broken

0.0%

IN299 708 OUT20 595

×

SimpleCalculateAverageBFCL/simple_python_82 (calculate_average): numbers=[12,15,18,20,21,26,30].1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 704 OUT459

×

SimpleCellDensityBFCL/simple_python_57 (calculate_cell_density): optical_density=0.6, dilution=5, calibration_factor omitted or 1e9.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 552 OUT752

×

SimpleCellInfoHumanBFCL/simple_python_55 (biology.get_cell_info): cell_type=human, detailed=true.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN33 620 OUT870

×

SimpleCellProteinsMembraneBFCL/simple_python_56 (cellbio.get_proteins): cell_compartment=plasma membrane, include_description omitted/true/false.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN28 998 OUT604

×

SimpleCityDistanceTrainBFCL/simple_python_86 (city_distance.find_shortest): start=New York, end=Los Angeles, transportation=train, allow_transfer=true.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 716 OUT621

×

SimpleEmployeeFetchDataBFCL/simple_python_90 (employee.fetch_data): company='ABC Ltd.', employee_id=345, data_field=[Personal Info, Job History].1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN29 475 OUT619

×

SimpleFetchDnaSequenceBFCL/simple_python_53 (fetch_DNA_sequence): DNA_id=DNA123, format omitted or 'fasta', upstream omitted or 0.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 367 OUT409

×

SimpleGetRestaurantSushiBFCL/simple_python_91 (get_restaurant): cuisine=sushi, location=Boston, condition=open on Sundays.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN20 054 OUT454

×

SimpleHeatCapacityAirBFCL/simple_python_52 (calc_heat_capacity): temp=298, volume=10, gas omitted or 'air'.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 806 OUT679

×

SimpleMolecularWeightBFCL/simple_python_59 (calculate_molecular_weight): compound=C6H12O6, to_unit=grams/mole or g/mol.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 512 OUT465

×

SimpleOrganelleFunctionLookupBFCL/simple_python_58 (cell_biology.function_lookup): molecule=ATP synthase, organelle=mitochondria, specific_function=true.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 537 OUT584

!

SimpleProteinSequenceBFCL/simple_python_54 (get_protein_sequence): gene=BRCA1, species omitted or 'Homo sapiens'.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Broken

IN24 367 OUT14 079

Tool Calling - Simple: Science, Data & Biology (set 4) 12 failed

0.0%

IN530 662 OUT10 273

×

SimpleAnalyzeDnaSequenceBFCL/simple_python_62 (analyze_dna_sequence): sequence + reference_sequence given, mutation_type omitted or 'substitution'.1 AI query 20 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 20

Failed

IN117 533 OUT2 136

×

SimpleDiabetesPredictionBFCL/simple_python_61 (diabetes_prediction): weight=150, height=70 (5ft10in), activity_level=lightly active.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN25 271 OUT760

×

SimpleFastestRouteAvoidTollsBFCL/simple_python_81 (map_routing.fastest_route): start=San Francisco/SF, end=Los Angeles/LA, avoid_tolls=true.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN25 190 OUT468

×

SimpleForestGrowthForecastBFCL/simple_python_68 (forest_growth_forecast): location=Yellowstone National Park, years=5, include_human_impact=true.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN50 476 OUT974

×

SimpleGeneticSimilarityBFCL/simple_python_63 (genetics.calculate_similarity): species1=Human, species2=Chimp/Chimpanzee, format omitted or 'percentage'.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN16 193 OUT291

×

SimpleGenotypeFrequencyBFCL/simple_python_64 (calculate_genotype_frequency): allele_frequency=0.3, genotype=AA.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN16 174 OUT322

×

SimpleIdentifyBirdBFCL/simple_python_67 (identify_bird): color=green, habitat=forest, size=small.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN16 138 OUT651

×

SimpleMutationTypeFindBFCL/simple_python_60 (mutation_type.find): snp_id=rs6034464, species omitted or 'Homo sapiens'.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN44 651 OUT843

×

SimplePopulationDensityBFCL/simple_python_65 (calculate_density): country=Brazil, year='2022', population=213000000, land_area=8500000.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN63 386 OUT1 163

×

SimplePrecipitationStatsBFCL/simple_python_66 (ecology_data.precipitation_stats): location=Amazon rainforest, time_frame=six_months.1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN85 393 OUT1 310

×

SimpleTurtlePopulationBFCL/simple_python_69 (ecology.get_turtle_population): location=Mississippi river, year=2020, species=true.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN45 136 OUT800

×

SimpleVehicleEmissionBFCL/simple_python_70 (calculate_vehicle_emission): vehicle_type=gas, miles_driven=1500, emission_factor omitted or 355.48.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN25 121 OUT555

Tool Calling - Simple: Sports & Games (set 1) 12 failed

0.0%

IN304 001 OUT7 372

×

SimpleBattingAverageComputeBFCL/simple_python_302 (calculate_batting_average): hits=180, at_bats=600, decimal_places=3 or omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 878 OUT319

×

SimpleBoardGameInfoTicketBFCL/simple_python_328 (boardgame.get_info): Ticket to Ride, {rating, player count} either order, language omitted/English.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN20 165 OUT464

×

SimpleFindCardInDeckBFCL/simple_python_335 (find_card_in_deck): rank Queen, suit Hearts, deck omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 615 OUT121

×

SimpleGameListByYearBFCL/simple_python_342 (game_list.get_games): year 2019, multiplayer true, ESRB Everyone.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 674 OUT594

×

SimpleGameWinnerByDateBFCL/simple_python_307 (game_result.get_winner): Lakers/Clippers either order, date 2021-01-28, venue omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN25 025 OUT384

×

SimpleHighestGameScoreBFCL/simple_python_349 (game_score.highest): Overwatch, PC, region Global or omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN24 042 OUT370

×

SimpleMatchScheduleBFCL/simple_python_308 (sports.match_schedule): Man United, 5 matches, English Premier League or omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 824 OUT391

×

SimpleNoteDurationBFCL/simple_python_300 (music.calculate_note_duration): 440/880 Hz, tempo omitted or 120.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 640 OUT977

×

SimplePokerGameWinnerMultiBFCL/simple_python_337 (poker_game_winner): 4 players + per-player hands dict, type omitted/Texas Holdem.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN31 256 OUT1 318

×

SimpleSoccerPlayerStatsBFCL/simple_python_303 (soccer_stat.get_player_stats): Cristiano Ronaldo, 2019-2020, league omitted.1 AI query 12 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN60 682 OUT1 225

×

SimpleSportRankingPlayerBFCL/simple_python_316 (get_sport_ranking): Tennis, Serena Williams, gender omitted/all/female.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 798 OUT509

×

SimpleTeamRankRegularBFCL/simple_python_317 (get_team_rank): LA Lakers, NBA, 2021, type regular.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN34 402 OUT700

Tool Calling - Simple: Sports & Games (set 2) 1 passed10 failed

9.1%

IN234 290 OUT7 345

×

SimpleBoardGameInfoCatanBFCL/simple_python_330 (board_game_info): Catan, {average_review_rating, age_range}.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 837 OUT505

×

SimpleCricketBattingScoreBFCL/simple_python_306 (average_batting_score): Virat Kohli, 10 matches, T20 or omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 754 OUT820

×

SimpleFifaTeamRankingBFCL/simple_python_318 (get_team_ranking): Germany, 2021, gender men or omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 796 OUT493

×

SimpleMatchResultsTwoTeamsBFCL/simple_python_325 (sports.match_results): Chicago Bulls vs Los Angeles Lakers, season omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 751 OUT378

×

SimplePokerDetermineWinnerTwoBFCL/simple_python_340 (card_games.poker_determine_winner): John vs Mike with 5-card hands (suit symbols).1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN60 279 OUT2 387

×

SimpleShuffleAndDrawBFCL/simple_python_336 (cards.shuffle_and_draw): num_cards 3.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 814 OUT149

×

SimpleSoccerPerformanceBFCL/simple_python_305 (sports_stats.get_performance): Messi, La Liga, 2020-2021, {goals scored, assists made}.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 143 OUT603

×

SimpleTeamScheduleGamesBFCL/simple_python_327 (sports_team.get_schedule): Man United, 6 games, Premier League, location omitted.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN32 068 OUT737

✓

SimpleThirdChordBFCL/simple_python_301 (get_third_chord): key C, type major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 106 OUT192

×

SimpleTopPlayerBySportBFCL/simple_python_323 (sports_ranking.get_top_player): tennis, gender women.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 706 OUT725

×

SimpleVideoGamePlayerStatsBFCL/simple_python_343 (game_stats.fetch_player_statistics): Zelda, user Sam, platform Switch.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 036 OUT356

Tool Calling - Simple: Sports & Games (set 3) 1 passed8 failed

11.1%

IN382 580 OUT7 364

×

SimpleCardDrawProbabilityBFCL/simple_python_338 (card_game_probability.calculate): total 52, desired 13 (hearts), cards_drawn 1 or omitted.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN52 999 OUT959

×

SimpleCelebrityNetWorthBFCL/simple_python_313 (celebrity_net_worth.get): Messi, currency EUR/euro.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 111 OUT398

×

SimpleDefenseRankingBFCL/simple_python_315 (get_defense_ranking): season 2021, top 1 or omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 406 OUT302

×

SimpleFindAthleteBFCL/simple_python_311 (sports_db.find_athlete): Lebron James, sport Basketball, team omitted.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN24 962 OUT476

×

SimpleGameItemStatsBFCL/simple_python_344 (get_game_item_stats): BotW, Guardian Sword+, stat power/power rating.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 922 OUT342

×

SimpleMonopolyOddsBFCL/simple_python_329 (monopoly_odds_calculator): number 7, dice_number 2, dice_faces 6 or omitted.1 AI query 11 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN47 756 OUT804

×

SimpleNflPlayerRecordBFCL/simple_python_309 (nfl_data.player_record): Tom Brady, 2020, team omitted or Tampa Bay Buccaneers.1 AI query 33 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 33

Failed

IN192 879 OUT3 063

✓

SimpleSportsRankingSeasonIntBFCL/simple_python_319 (sports_ranking): Man United, Premier League, season omitted or 2023.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 442 OUT579

×

SimpleTeamLatestScoreBFCL/simple_python_324 (team_score.get_latest): Los Angeles Lakers, include_opponent true.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 103 OUT441

Tool Calling - Simple: Sports & Games (set 4) 7 failed

0.0%

IN328 969 OUT7 707

×

SimpleCareerStatsBFCL/simple_python_310 (get_career_stats): LeBron James, team omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN12 376 OUT472

×

SimpleCelebrityAchievementsBFCL/simple_python_314 (sports_celebrity.get_major_achievements): Messi, sports Football/Soccer or omitted, team all or omitted.1 AI query 40 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 40

Failed

IN250 026 OUT3 616

×

SimpleChessTopPlayersBFCL/simple_python_331 (board_game.chess.get_top_players): New York, min rating 2300, number_of_players 10 or omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 160 OUT1 675

×

SimpleDeckOddsSuitBFCL/simple_python_341 (deck_of_cards.odds): suit hearts, deck_type without_joker or normal.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 134 OUT222

×

SimpleGameValuationBFCL/simple_python_345 (game_valuation): Super Mario Bros., 1985, condition Like New/New.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN19 709 OUT568

×

SimpleTeamPositionDetailedBFCL/simple_python_320 (sports_ranking.get_team_position): Golden State Warriors, 2022-2023, detailed true.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN16 074 OUT711

×

SimpleTeamScoreWithStatsBFCL/simple_python_326 (get_team_score): Los Angeles Lakers, NBA, include_player_stats any.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN12 490 OUT443

Tool Calling - Simple: Sports & Games (set 5) 7 failed

0.0%

IN162 740 OUT5 736

×

SimpleChessRatingBFCL/simple_python_332 (chess.rating): Magnus Carlsen, variant classical or omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN12 044 OUT264

×

SimpleCollectablesInSeasonBFCL/simple_python_346 (get_collectables_in_season): Animal Crossing: New Horizons, Spring, item_type all or omitted.1 AI query 14 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 14

Failed

IN58 709 OUT1 463

×

SimpleDetailedWeatherForecastBFCL/simple_python_333 (detailed_weather_forecast): London UK, 3 days, {high/low temp, humidity, precipitation}.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN32 456 OUT1 949

×

SimpleFullHouseProbabilityBFCL/simple_python_339 (poker_probability.full_house): defaults deck_size 52, hand_size 5.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN8 834 OUT516

×

SimplePlayerStatisticYearBFCL/simple_python_312 (player_statistic): Ronaldo, year 2021, team omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN19 010 OUT733

×

SimpleSoccerLastMatchBFCL/simple_python_347 (soccer.get_last_match): Liverpool F.C., include_stats true.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN8 852 OUT200

×

SimpleSportsRankingSeasonStrBFCL/simple_python_321 (sports_ranking): Barcelona, La Liga, 2021.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN22 835 OUT611

Tool Calling - Simple: Sports & Games (set 6) 1 passed3 failed

25.0%

IN101 392 OUT3 206

×

SimpleBlackjackWinnerBFCL/simple_python_334 (blackjack.check_winner): player [A,10], dealer [10,9], ace_value 1.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN17 322 OUT660

✓

SimpleCreatePlayerProfileBFCL/simple_python_348 (create_player_profile): StarPlayer, class Mage, starting_level 5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 061 OUT135

×

SimpleCurrentTeamRankingBFCL/simple_python_322 (sports_ranking.get_current): Liverpool, Premier League, season omitted or 2023-2024.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN20 023 OUT635

×

SimplePlayerLastGameStatsBFCL/simple_python_304 (player_stats.getLastGame): LeBron James, Lakers, metrics {Points, Rebounds}.1 AI query 13 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 13

Failed

IN58 986 OUT1 776

Tool Calling - Simple: Stats, Finance & ML (set 1) 8 failed

0.0%

IN200 420 OUT4 958

×

SimpleCardProbabilityBFCL/simple_python_112 (calculate_probability): total_outcomes=52, favorable_outcomes=4 (king from deck), round_to omitted or 2.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN24 940 OUT560

×

SimpleCompoundInterestNBFCL/simple_python_131 (calculate_compound_interest): principal=10000, rate=0.05, time=10, n=4.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN25 498 OUT787

×

SimpleInvestmentProfitBFCL/simple_python_134 (investment.predictProfit): investment_amount=5000, annual_return=0.07, years=5.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN33 755 OUT942

×

SimpleLightTravelTimeBFCL/simple_python_100 (light_travel_time): distance_in_light_years=4, speed_of_light omitted or default.1 AI query 10 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN41 775 OUT624

×

SimpleQuarterlyDividendBFCL/simple_python_128 (finance.calculate_quarterly_dividend_per_share): total_payout=50000000, outstanding_shares=100000000.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 514 OUT385

×

SimpleRandomForestTrainBFCL/simple_python_109 (random_forest.train): n_estimators=100, max_depth=5, data='my_data'.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 854 OUT238

×

SimpleStockPriceByDateBFCL/simple_python_142 (get_stock_price): company_name Amazon/AMZN, date=2022-03-11, exchange NASDAQ or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 095 OUT541

×

SimpleTTestStatsBFCL/simple_python_118 (stats.t_test): array_1=[10,15,12,14,11], array_2=[18,16,17,20,22], alpha=0.05.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN29 989 OUT881

Tool Calling - Simple: Stats, Finance & ML (set 2) 8 failed

0.0%

IN183 878 OUT5 934

×

SimpleCalculateSpeedBFCL/simple_python_101 (calculate_speed): distance=450, time=20, to_unit=km/h.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 839 OUT620

×

SimpleCardProbabilitySingleBFCL/simple_python_116 (probabilities.calculate_single): total_outcomes=52, event_outcomes=4 (king), round omitted or 2.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 938 OUT296

×

SimpleCompoundInterestFreqBFCL/simple_python_136 (compound_interest): principal=10000, annual_rate=5.0, compounding_freq=monthly, time_in_years=5.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 986 OUT643

×

SimpleDiscountedCashFlowBFCL/simple_python_129 (calculate_discounted_cash_flow): coupon_payment=100, period=5, discount_rate=0.04, face_value omitted or 1000.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN26 156 OUT997

×

SimpleHousePriceBedroomsBFCL/simple_python_110 (predict_house_price): bedrooms=3, bathrooms=2, area=1800, location San Francisco.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 048 OUT270

×

SimpleStockPriceDaysBFCL/simple_python_143 (get_stock_price): company=AAPL, days=5, exchange NASDAQ or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 742 OUT440

×

SimpleStockReturnProjectedBFCL/simple_python_137 (calculate_stock_return): investment_amount=5000, annual_growth_rate=0.06, holding_period=5, dividends omitted or false.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN49 790 OUT1 065

×

SimpleTTestIndBFCL/simple_python_119 (hypothesis_testing.ttest_ind): sample1=[22,33,42,12,34], sample2=[23,45,44,14,38], significance_level=0.05.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 379 OUT1 603

Tool Calling - Simple: Stats, Finance & ML (set 3) 1 passed7 failed

12.5%

IN190 461 OUT4 368

×

SimpleCompoundedInterestAnnuallyBFCL/simple_python_145 (calculate_compounded_interest): principal=5000, interest_rate=0.05, period=10, compounding_frequency omitted or Annually.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN25 427 OUT676

×

SimpleDistanceEarthMoonBFCL/simple_python_102 (calculate_distance): body1=Earth, body2=Moon, unit mi/miles/mile.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 101 OUT419

✓

SimpleHeartRatioProbabilityBFCL/simple_python_117 (probability_of_event): success_outcomes=13 (hearts), total_outcomes=52, format_as_ratio=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 515 OUT171

×

SimpleHousePriceRoomsBFCL/simple_python_125 (predict_house_price): area=2500, rooms=5, year=1990, location San Francisco/SF.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 434 OUT427

×

SimpleMutualFundReturnBFCL/simple_python_139 (estimate_mutual_fund_return): yearly_yield=5.0, investment_amount=2000, years=3.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN29 784 OUT760

×

SimpleReturnOnEquityBFCL/simple_python_132 (calculate_return_on_equity): net_income=2000000, shareholder_equity=10000000, dividends_paid=200000.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN34 928 OUT843

×

SimpleStockPriceCurrentMultiBFCL/simple_python_149 (get_stock_price): company_names=[Apple, Microsoft] (or tickers AAPL, MSFT).1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN20 924 OUT322

×

SimpleTwoSampleTTestRunBFCL/simple_python_120 (run_two_sample_ttest): group1=[3,4,5,6,4], group2=[7,8,9,8,7], equal_variance=true.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN30 348 OUT750

Tool Calling - Simple: Stats, Finance & ML (set 4) 1 passed7 failed

12.5%

IN151 252 OUT4 676

×

SimpleBinomialProbDistBFCL/simple_python_114 (prob_dist.binomial): trials=10, successes=5, p omitted or 0.5.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 422 OUT930

×

SimpleFutureValueMonthlyBFCL/simple_python_133 (finance.predict_future_value): present_value=5000, annual_interest_rate=0.05, compounding_periods_per_year=12, time_years=3.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN25 995 OUT807

×

SimpleGeometryAreaTriangleBFCL/simple_python_104 (geometry.area_triangle): base=6, height=10, unit omitted or 'square meters'.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 594 OUT227

✓

SimpleMathPowerBFCL/simple_python_105 (math.power): base=3, exponent=4, mod omitted or 1.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 554 OUT120

×

SimpleNPVCashFlowsBFCL/simple_python_127 (calculate_NPV): cash_flows=[200,300,400,500], discount_rate=0.1, initial_investment=2000.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 911 OUT930

×

SimpleReturnOnInvestmentBFCL/simple_python_135 (calculate_return_on_investment): purchase_price=20, sale_price=25, dividend=2.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 526 OUT438

×

SimpleStockPriceDataTypeBFCL/simple_python_146 (stock_price): company Amazon/AMZN, days=3, data_type omitted or Close.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 332 OUT325

×

SimpleTwoSampleTTestFloatBFCL/simple_python_123 (hypothesis_testing.two_sample_t_test): group1=[12.4,15.6,11.2,18.9], group2=[10.5,9.8,15.2,13.8], alpha omitted or 0.05.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN26 918 OUT899

Tool Calling - Simple: Stats, Finance & ML (set 5) 8 failed

0.0%

IN756 286 OUT12 864

×

SimpleAreaUnderCurvePolyBFCL/simple_python_103 (mathematics.calculate_area_under_curve): polynomial=[3.0,2.0,-4.0], limits=[-1.0,2.0].1 AI query 29 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 29

Failed

IN168 057 OUT2 980

×

SimpleBinomialCalcBFCL/simple_python_115 (calculate_binomial_probability): number_of_trials=8, number_of_successes=5, probability_of_success omitted or 0.5.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN27 488 OUT562

×

SimpleCalculateBmiBFCL/simple_python_107 (calculate_bmi): weight=70, height=175, system omitted or metric.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 828 OUT244

×

SimpleFutureValueInvestmentBFCL/simple_python_148 (finance.calculate_future_value): initial_investment=20000, rate_of_return=0.08, years=5, contribution omitted or 0.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN26 037 OUT504

×

SimpleLinearRegressionBFCL/simple_python_108 (run_linear_regression): predictors=[Age,Income,Education], target=Purchase_Amount, standardize=true.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 894 OUT308

×

SimpleNPVInvestmentBFCL/simple_python_130 (finance_calculator.npv): cash_flows=[-50000,10000,15000,20000,25000,30000], discount_rate=0.08, years omitted or empty.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN467 126 OUT6 862

×

SimpleStockPricesDurationBFCL/simple_python_147 (get_stock_prices): companies=[Microsoft,Google], duration='2 weeks'.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN22 035 OUT356

×

SimpleTTestDatasetsBFCL/simple_python_124 (t_test): dataset_A=[12,24,36], dataset_B=[15,30,45], alpha omitted or 0.05.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 821 OUT1 048

Tool Calling - Simple: Stats, Finance & ML (set 6) 2 passed8 failed

20.0%

IN619 040 OUT10 088

✓

SimpleBinomialCoin100BFCL/simple_python_121 (calc_binomial_prob): num_trials=100, num_success=60, prob_success=0.5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 157 OUT140

×

SimpleCagrBFCL/simple_python_140 (calculate_cagr): initial_value=2000, final_value=3000, period_in_years=4.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN388 047 OUT4 480

×

SimpleChiSquaredTestBFCL/simple_python_122 (chi_squared_test): table=[[10,20],[30,40]], alpha omitted or 0.05.1 AI query 7 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN31 988 OUT574

✓

SimpleDiceRollBFCL/simple_python_113 (probability.dice_roll): desired_number=6, number_of_rolls=2, die_sides omitted or 6.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 134 OUT126

×

SimpleMarketPerformanceBFCL/simple_python_144 (market_performance.get_data): indexes=[S&P 500, Dow Jones], days=5, detailed omitted or true/false.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN47 256 OUT927

×

SimpleMetalPriceBFCL/simple_python_141 (get_metal_price): metal=Gold, measure=ounce.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN31 610 OUT484

×

SimpleNormalVariateBFCL/simple_python_111 (random.normalvariate): mu=0, sigma=1.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 844 OUT264

×

SimplePortfolioFutureValueBFCL/simple_python_138 (portfolio_future_value): stock=X, invested_amount=5000, expected_annual_return=0.05, years=7.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN47 528 OUT989

×

SimpleRandomForestClassifierBFCL/simple_python_106 (train_random_forest_classifier): dataset=your_dataset_name, max_depth=5, n_estimators=100.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 941 OUT355

×

SimpleRSquaredBFCL/simple_python_126 (linear_regression.get_r_squared): dataset_path=C:/data/cars.csv, independent_variables=[engine_size,fuel_economy], dependent_variable=car_price.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN28 535 OUT1 749

Tool Calling - Tool Selection: Area (Triangle/Circle) 1 failed

0.0%

IN15 215 OUT543

×

SelectTriangleAreaHeronToolBFCL/multiple_1 (math.triangle_area_heron): triangle area from three side lengths (Heron).1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN15 215 OUT543

Tool Calling - Tool Selection: Art & Music 9 failed

0.0%

IN555 295 OUT6 746

×

SelectAnalyzeStructureToolBFCL/multiple_162 (analyze_structure): dynamic structural analysis of specific floors.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN24 073 OUT328

×

SelectConcertSearchToolBFCL/multiple_166 (concert.search): concert search by genre/location/date/price.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN24 013 OUT464

×

SelectGuitarPriceToolBFCL/multiple_165 (guitar_price.find): guitar price by model/condition/location (not instrument_price.get).1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN40 829 OUT575

×

SelectInstrumentPriceToolBFCL/multiple_164 (instrument_price.get): instrument price by brand/model/finish (not guitar_price.find).1 AI query 14 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 14

Failed

IN116 594 OUT1 124

×

SelectMelodyGeneratorToolBFCL/multiple_167 (music_generator.generate_melody): melody generation with key/note/length/tempo. Tightened per methodology: prompt explicitly gives "120 beats per minute", so tempo is required exactly 120 (ground truth also accepts omission, which equals the schema default).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN30 530 OUT658

×

SelectMetropolitanArtworksToolBFCL/multiple_163 (metropolitan_museum.get_top_artworks): top artworks sorted by popularity.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN36 814 OUT806

×

SelectMusicalScaleToolBFCL/multiple_169 (musical_scale): musical scale for a key.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN42 484 OUT367

×

SelectSculptureInfoToolBFCL/multiple_160 (get_sculpture_info): sculpture info by artist with detail.1 AI query 21 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 21

Failed

IN160 913 OUT1 665

×

SelectSongLyricsToolBFCL/multiple_168 (get_song_lyrics): lyrics by song title/artist.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN79 045 OUT759

Tool Calling - Tool Selection: Calculators & Directories 8 failed

0.0%

IN316 581 OUT4 421

×

SelectCalculateGcdToolBFCL/multiple_35 (calculate_gcd): greatest common divisor, not lcm.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN19 701 OUT285

×

SelectCurrencyGetRateToolBFCL/multiple_17 (currency_conversion.get_rate): rate at a specific date, not convert/latest-rate.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN36 243 OUT580

×

SelectGeodistanceToolBFCL/multiple_65 (geodistance.find): distance between cities, not timezone difference/flights.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN19 676 OUT216

×

SelectLibrarySearchBookToolBFCL/multiple_38 (library.search_book): search for a book, not reserve one.1 AI query 20 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 20

Failed

IN129 860 OUT1 506

×

SelectLinearRegressionToolBFCL/multiple_53 (linear_regression): linear regression, not random forest.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN25 137 OUT404

×

SelectMagneticFieldCalculateToolBFCL/multiple_41 (magnetic_field.calculate): magnetic field via Ampere's law, not electric field/force.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN19 739 OUT535

×

SelectSolveQuadraticEquationToolBFCL/multiple_28 (solve.quadratic_equation): solve a quadratic, not rgb/string-reverse.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN30 427 OUT404

×

SelectStockForecastToolBFCL/multiple_55 (stock_forecast): stock price prediction, not weather forecast.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN35 798 OUT491

Tool Calling - Tool Selection: Finance & Legal 10 failed

0.0%

IN1 079 269 OUT12 212

×

SelectCagrCalculationToolBFCL/multiple_130 (calculate_cagr): compound annual growth rate from initial/final/years, not other finance tools.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN56 271 OUT676

×

SelectCaseInfoToolBFCL/multiple_135 (get_case_info): retrieve case info (victim) by docket/court.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN71 471 OUT879

×

SelectCrimeRateToolBFCL/multiple_136 (get_crime_rate): crime rate by city/state/type/year.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN48 618 OUT619

×

SelectCrimeRecordToolBFCL/multiple_134 (crime_record.get_record): detailed crime record by case number/county.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN71 617 OUT753

×

SelectFutureValueToolBFCL/multiple_132 (finance.calculate_future_value): future value from rate/initial/years, not compound_interest.1 AI query 7 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN56 355 OUT770

×

SelectLawsuitDetailsToolBFCL/multiple_139 (lawsuit_details.find): patent lawsuit details by company/year/type.1 AI query 32 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 32

Failed

IN294 258 OUT2 909

×

SelectLawsuitSearchToolBFCL/multiple_137 (lawsuit_search): lawsuit search by company/date/location/status. Tightened per methodology: prompt explicitly asks for "ongoing" lawsuits, so status is required exactly "ongoing" (ground truth also accepts omission, but an explicitly-stated value must be passed).1 AI query 32 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 32

Failed

IN295 238 OUT2 786

×

SelectLegalCaseFetchToolBFCL/multiple_138 (legal_case.fetch): fetch detailed legal case by id.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN72 645 OUT1 149

×

SelectMarketPerformanceToolBFCL/multiple_131 (market_performance.get_data): index performance over N days.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN64 122 OUT816

×

SelectMutualFundBalanceToolBFCL/multiple_133 (calculate_mutual_fund_balance): mutual fund balance from investment/yield/years.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN48 674 OUT855

Tool Calling - Tool Selection: Finance & Markets 8 failed

0.0%

IN908 136 OUT11 878

×

SelectAvgClosingPriceToolBFCL/multiple_56 (avg_closing_price): average closing price, not volume/total revenue.1 AI query 47 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 47

Failed

IN408 370 OUT3 840

×

SelectCompoundInterestCalcToolBFCL/multiple_49 (calc_Compound_Interest): compound interest, not future value/simple interest.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN44 879 OUT649

×

SelectCurrencyConversionToolBFCL/multiple_52 (currency_conversion): currency conversion, not unit conversion.1 AI query 21 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 21

Failed

IN181 383 OUT1 819

×

SelectDividendDataToolBFCL/multiple_54 (corporate_finance.dividend_data): dividend history, not generic stock market data.1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN106 040 OUT1 375

×

SelectHousePriceForecastToolBFCL/multiple_50 (house_price_forecast): house price forecast, not weather/stock forecast.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN44 251 OUT707

×

SelectPropertyDepreciationToolBFCL/multiple_14 (finance.property_depreciation): depreciation, not loan/inflation.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN44 563 OUT1 030

×

SelectRevenueForecastToolBFCL/multiple_13 (corporate_finance.revenue_forecast): revenue forecast, not product price.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN51 330 OUT1 866

×

SelectYoyGrowthRateToolBFCL/multiple_12 (corporate_finance.calculate_YOY_growth_rate): YOY growth, not ROE/ROA.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN27 320 OUT592

Tool Calling - Tool Selection: Food & Travel 1 passed6 failed

14.3%

IN283 076 OUT5 037

×

SelectCurrencyConvertToolBFCL/multiple_95 (currency_exchange.convert): currency conversion, not unit_conversion. live_conversion is required true: prompt says "using current exchange rate" (ground truth = [true]).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN52 244 OUT771

×

SelectDrivingDistanceToolBFCL/multiple_104 (get_shortest_driving_distance): driving distance, not instrument price / biology / scientist lookup.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN29 344 OUT476

✓

SelectHotelBookToolBFCL/multiple_93 (hotel.book): book a hotel room, not car.rental.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 245 OUT117

×

SelectRecipeFindToolBFCL/multiple_91 (recipe.find): recipe by main ingredient + limit, not restaurant.find.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN34 554 OUT618

×

SelectRecipeSearchToolBFCL/multiple_89 (recipe_search): find a recipe from ingredients, not restaurant_search/ingredient_replace.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN46 956 OUT963

×

SelectRestaurantForGroupToolBFCL/multiple_90 (restaurant.find_group): restaurant for a group, not events.find_event.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN58 156 OUT928

×

SelectWalmartVeganToolBFCL/multiple_92 (walmart.vegan_products): the named store is Walmart, not Safeway/Whole Foods.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN52 577 OUT1 164

Tool Calling - Tool Selection: Games, Food & Utilities 10 failed

0.0%

IN745 095 OUT8 740

×

SelectBestGroceryStoreToolBFCL/multiple_188 (grocery_store.find_best): best grocery store by rating/products.1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN84 066 OUT983

×

SelectClosestRestaurantToolBFCL/multiple_185 (restaurant_search.find_closest): closest restaurant by cuisine/amenities.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN47 075 OUT430

×

SelectFindRecipeToolBFCL/multiple_186 (find_recipe): recipe by restrictions/type/time.1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN136 845 OUT1 557

×

SelectGamePlayerStatsToolBFCL/multiple_180 (game_stats.fetch_player_statistics): player stats by game/username/platform.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN35 500 OUT429

×

SelectMultiplayerGameFinderToolBFCL/multiple_182 (multiplayer_game_finder): multiplayer game by platform/rating.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN42 988 OUT962

×

SelectRecipeCaloriesToolBFCL/multiple_183 (recipe_info.get_calories): calories of a recipe from a website.1 AI query 14 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 14

Failed

IN117 845 OUT1 307

×

SelectRecipeSearchToolBFCL/multiple_184 (recipe_search): recipe by restriction/ingredients/servings.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN50 869 OUT826

×

SelectSoccerLastMatchToolBFCL/multiple_181 (soccer.get_last_match): last match with stats for a club.1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN135 593 OUT1 099

×

SelectTimezoneConvertToolBFCL/multiple_189 (timezone.convert): convert a time between timezones.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN43 076 OUT583

×

SelectWholeFoodsPriceToolBFCL/multiple_187 (whole_foods.check_price): price of items at a Whole Foods location.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN51 238 OUT564

Tool Calling - Tool Selection: Geometry, Finance & Services 7 failed

0.0%

IN272 755 OUT4 477

×

SelectAirQualityForecastToolBFCL/multiple_63 (air_quality_forecast): air quality, not weather/news.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN64 503 OUT740

×

SelectFinancialCompoundInterestToolBFCL/multiple_57 (financial.compound_interest): compound interest, not simple interest.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN37 615 OUT926

×

SelectGeometryRectangleToolBFCL/multiple_31 (geometry_rectangle.calculate): rectangle area+perimeter, not square/circle.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN12 805 OUT319

×

SelectLawyerFinderToolBFCL/multiple_59 (lawyer_finder): find a lawyer by specialization, not rent a car.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN31 906 OUT474

×

SelectLibrarySearchBooksToolBFCL/multiple_68 (library.search_books): search a specific library, not Google Books/Open Library.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN12 796 OUT153

×

SelectMapsShortestPathToolBFCL/multiple_27 (maps.shortest_path): shortest path with transport mode, not route-time estimation.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN53 669 OUT943

×

SelectTrafficEstimateToolBFCL/multiple_66 (traffic_estimate): traffic for a time period, not raw distance/weather.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN59 461 OUT922

Tool Calling - Tool Selection: History & Knowledge 1 passed9 failed

10.0%

IN327 030 OUT6 296

×

SelectDiscovererToolBFCL/multiple_155 (get_discoverer): discoverer of a discovery, with detail.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN20 528 OUT316

×

SelectEarliestReferenceToolBFCL/multiple_157 (get_earliest_reference): earliest reference of a person from a source.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN27 652 OUT456

×

SelectEventDateToolBFCL/multiple_153 (get_event_date): date of a historical event.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN27 605 OUT167

×

SelectHistoricalContribToolBFCL/multiple_156 (historical_contrib.get_contrib): scientist's contribution on a date.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN50 237 OUT1 573

×

SelectHistoryKeyEventsToolBFCL/multiple_152 (history.get_key_events): key war events by country/years.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN41 090 OUT1 230

×

SelectMostFollowedToolBFCL/multiple_151 (social_media_analytics.most_followed): most-followed by topic/sub-topics.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN40 959 OUT642

×

SelectNeuronalActivityToolBFCL/multiple_150 (calculate_neuronal_activity): neuronal activity from synaptic rate/weight/decay.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN27 817 OUT344

×

SelectPaintNeededToolBFCL/multiple_159 (calculate_paint_needed): paint quantity for a wall.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN35 120 OUT387

×

SelectPapalBiographyToolBFCL/multiple_158 (religious_history.get_papal_biography): papal biography with contributions.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN42 421 OUT1 037

✓

SelectUsPresidentToolBFCL/multiple_154 (US_president.in_year): president in a given year. Tightened per methodology: prompt explicitly asks for the "full name", so full_name is required true (ground truth also accepts omission, but an explicitly-requested option must be passed).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 601 OUT144

Tool Calling - Tool Selection: History, Demographics & Art 3 passed5 failed2 skipped

37.5%

IN366 387 OUT51 934

✓

SelectArtworkPriceToolBFCL/multiple_74 (art_auction.fetch_artwork_price): artwork price, not library book search. Tightened per methodology: prompt explicitly asks "on all auction platforms", so require platform == "all" (ground truth also accepts omission, but an explicitly-given value must pass).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 768 OUT275

×

SelectArtworkSearchToolBFCL/multiple_77 (artwork_search.find): artwork search, not tourist_attraction/park search.1 AI query 499 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 499

Failed

IN156 874 OUT47 810

–

SelectCustomSculptureToolBFCL/multiple_76 (sculpture.create_custom): custom sculpture (material), not painting (color).0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Skipped: Ambiguous action-vs-question item (item flaw, not model behavior): the prompt mixes an order ('I want to order a custom bronze sculpture of a horse') with a question ('What material options are available?'), so a competent model may either call sculpture_create_custom or answer the question in text - not deterministic. Kept for traceability.

Skipped

IN0 OUT0

×

SelectEuropeanMonarchsToolBFCL/multiple_70 (european_history.get_monarchs): monarch lookup, not events/culture.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN46 239 OUT695

×

SelectExhibitionInfoToolBFCL/multiple_79 (exhibition_info): museum exhibitions, not restaurant_info.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN16 994 OUT1 322

–

SelectMuseumInfoToolBFCL/multiple_78 (museum_info): museum info with a feature list, not generic tourist_spot_info.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Ambiguous single-call item (item flaw, not model behavior): a dedicated exhibition_info tool coexists with museum_info, so the 'exhibitions' part can reasonably route either to museum_info's features or to a separate exhibition_info call; a competent model may make one or two calls - not deterministic. Kept for traceability.

Skipped

IN0 OUT0

×

SelectPaintColorTrendsToolBFCL/multiple_75 (paint_color.trends): paint color trend, not weather/house-price trends.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN54 708 OUT562

×

SelectPopulationByCategoryToolBFCL/multiple_71 (get_population): population by category, not bureau statistics/demographics.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN59 312 OUT656

✓

SelectReligionOriginToolBFCL/multiple_73 (religion.get_origin): founder/origin, not core beliefs.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 741 OUT322

✓

SelectStatePopulationToolBFCL/multiple_72 (us_history.population_by_state_year): state population, not GDP.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 751 OUT292

Tool Calling - Tool Selection: Isolated Collisions: Exhibition, Hotel & Currency 2 passed1 failed

66.7%

IN42 421 OUT2 676

✓

SelectBookLuxuryHotelToolBFCL/multiple_191 (book_hotel): luxury room with view, stay_duration variant.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 065 OUT208

×

SelectCurrencyConversionToolBFCL/multiple_192 (currency_conversion.convert): convert an amount between currencies.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN28 351 OUT1 284

✓

SelectExhibitionToolBFCL/multiple_161 (find_exhibition): top-rated sculpture exhibition by location/art_form.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 005 OUT1 184

Tool Calling - Tool Selection: Knowledge & Media 10 failed

0.0%

IN288 062 OUT6 230

×

SelectBattleDetailsToolBFCL/multiple_18 (european_history.battle_details): battle details, not war/leader info.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN50 105 OUT558

×

SelectDatabaseModifyColumnsToolBFCL/multiple_10 (database.modify_columns): delete columns, not create a backup.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN37 151 OUT426

×

SelectFiveFactorModelToolBFCL/multiple_69 (five_factor_model.analyse): Big Five personality, not MBTI.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN37 312 OUT571

×

SelectGameRewardsToolBFCL/multiple_26 (game_rewards.get): rewards, not scores/missions list.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN25 689 OUT1 591

×

SelectGenerateSoundWaveToolBFCL/multiple_21 (generate_sound_wave): generate a sound, not play a file.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN17 822 OUT512

×

SelectMostPointsSingleGameToolBFCL/multiple_22 (sports_data.basketball.most_points_single_game): single-game record, not season/career.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN23 963 OUT271

×

SelectPlayerStatsToolBFCL/multiple_23 (basketball.player_stats.get): player stats, not game/team stats.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN36 370 OUT504

×

SelectReligionSchismsToolBFCL/multiple_19 (religion_history.get_schisms): schisms, not councils/reformations.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN23 971 OUT441

×

SelectSculpturePriceToolBFCL/multiple_20 (sculpture_price.calculate): commission price, not sculptor info/availability.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN17 810 OUT1 007

×

SelectVideoGameStorePriceToolBFCL/multiple_25 (video_games.store_price): store price, not store currency/on-sale.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN17 869 OUT349

Tool Calling - Tool Selection: Legal, Environment & Services 1 passed9 failed

10.0%

IN1 097 747 OUT11 321

×

SelectAirQualityToolBFCL/multiple_144 (air_quality): AQI by location/date.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN38 083 OUT487

×

SelectEmissionsToolBFCL/multiple_145 (calculate_emissions): annual CO2 from distance/fuel/efficiency.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN52 332 OUT827

×

SelectHumidityForecastToolBFCL/multiple_142 (weather.humidity_forecast): humidity forecast by location/days.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN28 605 OUT286

×

SelectLawsuitCheckCaseToolBFCL/multiple_141 (lawsuit.check_case): verify a case's closed status by id.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN73 795 OUT763

×

SelectLawsuitsSearchToolBFCL/multiple_140 (lawsuits_search): lawsuits by company/location/year.1 AI query 45 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 45

Failed

IN429 354 OUT3 930

×

SelectMapDirectionsToolBFCL/multiple_147 (map_service.get_directions): directions avoiding features.1 AI query 11 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN82 106 OUT953

×

SelectRestaurantFindNearbyToolBFCL/multiple_146 (restaurant.find_nearby): nearby restaurants by cuisine/distance.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN66 660 OUT789

✓

SelectSentimentAnalysisToolBFCL/multiple_149 (sentiment_analysis): sentiment of a review text.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 162 OUT107

×

SelectSlopeGradientToolBFCL/multiple_143 (calculate_slope_gradient): slope gradient between two coordinates. Tightened per methodology: prompt explicitly asks for the gradient "in degree", so unit is required exactly "degree" (ground truth also accepts omission, which equals the schema default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN35 706 OUT481

×

SelectStockInfoToolBFCL/multiple_148 (get_stock_info): detailed stock information by company.1 AI query 32 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 32

Failed

IN278 944 OUT2 698

Tool Calling - Tool Selection: Local Services & Weather 9 failed

0.0%

IN431 213 OUT5 735

×

SelectHumidityTemperatureForecastToolBFCL/multiple_60 (humidity_temperature_forecast): humidity/temperature, not events/movies.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN28 918 OUT263

×

SelectLandscapeArchitectToolBFCL/multiple_61 (landscape_architect.find_specialty): landscape architect, not home renovation.1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN101 899 OUT1 220

×

SelectLawyerSearchToolBFCL/multiple_58 (lawyer.search): find a lawyer, not a doctor.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN66 103 OUT743

×

SelectLibraryFindNearbyToolBFCL/multiple_48 (library.find_nearby): find a library, not a store.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN70 670 OUT1 067

×

SelectNatureParkToolBFCL/multiple_62 (nature_park.find_nearby): find a nature park, not a restaurant.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN73 121 OUT1 035

×

SelectRideHailingToolBFCL/multiple_39 (ride_hailing.get_rides): find a ride, not a grocery delivery.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN19 807 OUT327

×

SelectRoutePlannerToolBFCL/multiple_24 (route_planner.calculate_route): plan a route, not chess-club lookup.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN19 839 OUT421

×

SelectTranslateToolBFCL/multiple_67 (translate): translate text, not sentiment/word-count.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN14 635 OUT169

×

SelectUvIndexToolBFCL/multiple_64 (uv_index.get_future): UV index, not rainfall/snowfall.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN36 221 OUT490

Tool Calling - Tool Selection: Mathematics 8 failed

0.0%

IN329 455 OUT4 900

×

SelectCalculateIntegralToolBFCL/multiple_33 (calculate_integral): definite integral, not derivative.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN38 105 OUT572

×

SelectCosineSimilarityToolBFCL/multiple_47 (cosine_similarity.calculate): cosine similarity, not correlation.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN47 355 OUT1 148

×

SelectDiceRollProbabilityToolBFCL/multiple_51 (dice_roll_probability): dice sum probability, not coin/card.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN19 339 OUT368

×

SelectFunctionsIntersectToolBFCL/multiple_29 (functions.intersect): intersection of two functions, not zeros of one.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN71 490 OUT908

×

SelectMathLcmToolBFCL/multiple_34 (math.lcm): least common multiple, not gcd/sqrt.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN46 705 OUT421

×

SelectQuadraticRootsToolBFCL/multiple_11 (math_roots.quadratic): quadratic roots from coefficients, not cubic/polynomial.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN35 395 OUT658

×

SelectRectangleAreaToolBFCL/multiple_30 (rectangle.area): rectangle area, not circle/triangle area.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN24 410 OUT246

×

SelectSortListToolBFCL/multiple_46 (sort_list): sort a list, not filter/sum.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN46 656 OUT579

Tool Calling - Tool Selection: Mathematics & Geometry 2 passed5 failed

28.6%

IN152 140 OUT2 317

✓

SelectCircleAreaToolBFCL/multiple_97 (geometry.area_circle): circle area, not plot_sine_wave.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 030 OUT126

×

SelectCircumferenceToolBFCL/multiple_98 (geometry.circumference): circle circumference, not the reference/time/melody tools.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN37 261 OUT512

×

SelectDatabaseQueryToolBFCL/multiple_119 (database.query): structured DB query with conditions, not the score/ROI/probability distractors.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN38 382 OUT657

×

SelectDerivativeToolBFCL/multiple_99 (calculus.derivative): derivative at a point, not get_personality_traits.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN20 623 OUT324

✓

SelectHcfToolBFCL/multiple_100 (math.hcf): highest common factor, not the scale-progression distractor.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 040 OUT78

×

SelectSolveQuadraticToolBFCL/multiple_96 (solve_quadratic_equation): solve quadratic, not stock/portfolio tools.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN15 253 OUT398

×

SelectTriangleAreaToolBFCL/multiple_121 (geometry.area_triangle): triangle area, not map directions / invention lookup.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN20 551 OUT222

Tool Calling - Tool Selection: Mixed Domains 7 failed1 skipped

0.0%

IN210 862 OUT4 073

×

SelectAverageGradeToolBFCL/multiple_9 (calculate_average): selects average, not standard_deviation/highest_grade.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN36 859 OUT414

×

SelectCapacitanceToolBFCL/multiple_6 (capacitance_calculator.calculate): selects capacitance, not resistance/magnetic_field. Tightened per methodology: prompt explicitly states "dielectric constant K is 1.0", so K is required exactly 1.0 (ground truth also accepts omission, but an explicitly-given value must be passed).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN25 690 OUT637

×

SelectCapitalToolBFCL/multiple_2 (country_info.capital): selects capital lookup, not largest_city/population.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN19 949 OUT146

×

SelectDisplacementToolBFCL/multiple_4 (kinematics.calculate_displacement): selects displacement, not final_speed.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN24 097 OUT666

×

SelectEuclideanDistanceToolBFCL/multiple_3 (EuclideanDistance.calculate): selects distance, not angleToXAxis.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN29 462 OUT468

×

SelectRealEstateSearchToolBFCL/multiple_8 (realestate.find_properties): property search with budget, not property_valuation.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN37 499 OUT696

×

SelectWeatherByCoordinatesDateToolBFCL/multiple_5 (weather.get_by_coordinates_date): historical-by-coordinates, not by-city/forecast.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN37 306 OUT1 046

–

SelectWildlifeGrowthToolBFCL/multiple_7 (wildlife_population.assess_growth): population growth, not ecological_impact.0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Skipped: Ambiguous single-call item (item flaw, not model behavior): the prompt asks for BOTH population growth AND impact on woodland, and two distinct tools genuinely fit (wildlife_population_assess_growth + ecological_impact_analyze); a competent human would reasonably make two calls, so a single-call ground truth is unfair. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Tool Selection: Music, Sports & Games 9 failed

0.0%

IN392 190 OUT5 787

×

SelectBoardGameRecommendToolBFCL/multiple_86 (BoardGameGeek.recommend): BoardGameGeek store, not AmazonGameStore.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN26 046 OUT625

×

SelectConcertTicketToolBFCL/multiple_81 (concert.book_ticket): concert ticket, not festival ticket.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN15 259 OUT213

×

SelectGameUpdateToolBFCL/multiple_87 (games.update.find): game update/patch, not reviews/price.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN31 740 OUT482

×

SelectMusicGenerateToolBFCL/multiple_82 (music.generate): musical piece from key/tempo, not raw audio.generate.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN15 181 OUT127

×

SelectMusicShopToolBFCL/multiple_80 (music_shop.find_nearby): music shop with lessons, not gym.1 AI query 17 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 17

Failed

IN114 100 OUT1 600

×

SelectPlayerCountToolBFCL/multiple_88 (video_games.get_player_count): active player count, not sales.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN31 687 OUT426

×

SelectPlayerGoalsToolBFCL/multiple_83 (player_stats.get_all_time_goals): per-player goals, not team/league top scorer.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN31 732 OUT486

×

SelectSoccerScoresToolBFCL/multiple_85 (soccer_scores.get_scores): soccer scores, not basketball.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN43 535 OUT636

×

SelectTopGoalScorersToolBFCL/multiple_84 (getTopGoalScorers): top goal scorers, not top assists.1 AI query 13 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 13

Failed

IN82 910 OUT1 192

Tool Calling - Tool Selection: Physical Sciences 10 failed

0.0%

IN451 706 OUT8 382

×

SelectBiologicalEnergyToolBFCL/multiple_43 (biological.calc_energy): energy from moles of substance, not biomass/work.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN48 625 OUT667

×

SelectBiotSavartLawToolBFCL/multiple_40 (electromagnetism.biot_savart_law): magnetic field via Biot-Savart, not Ampere's law.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN48 513 OUT768

×

SelectConeVolumeToolBFCL/multiple_32 (geometry.calculate_cone_volume): cone volume, not cone mass.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN29 179 OUT466

×

SelectFinalTemperatureToolBFCL/multiple_42 (calculate_final_temperature): mixing two gas samples, not mass calculation.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN35 803 OUT649

×

SelectFinalVelocityToolBFCL/multiple_37 (kinematics.final_velocity): final velocity from initial velocity + time, not wave velocity/distance.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN17 131 OUT381

×

SelectGeologyEraToolBFCL/multiple_45 (geology.get_era): geological era date, not historical event date.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN39 293 OUT611

×

SelectPopulationGeneticsNeToolBFCL/multiple_16 (population_genetics.calculate_ne): effective population size, not range-shift/carrying-capacity.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN33 329 OUT936

×

SelectSolarFarmPotentialToolBFCL/multiple_15 (solarFarm.potential): solar (not wind) energy potential from coordinates + panel area.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN70 957 OUT1 709

×

SelectSpeedFromRestToolBFCL/multiple_36 (kinematics.calculate_speed_from_rest): speed from rest over a distance, not acceleration.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN42 031 OUT1 114

×

SelectWeightInSpaceToolBFCL/multiple_44 (calculate.weight_in_space): weight on a planet, not unit/currency conversion.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN86 845 OUT1 081

Tool Calling - Tool Selection: Physics & Biology 13 failed

0.0%

IN986 042 OUT10 776

×

SelectCellProteinsToolBFCL/multiple_109 (cellbio.get_proteins): proteins by cell compartment, not find_exhibition / mutation lookup.1 AI query 20 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 20

Failed

IN189 818 OUT2 147

×

SelectDensityToolBFCL/multiple_107 (calculate_density): mass/volume density, not mix_paint_color.1 AI query 19 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 19

Failed

IN176 010 OUT1 370

×

SelectDisplacementToolBFCL/multiple_102 (calculate_displacement): kinematic displacement, not poker/musical distractors.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN55 044 OUT630

×

SelectElectricFieldToolBFCL/multiple_106 (calculate_electric_field_strength): electric field, not paint/cooking/group distractors.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN46 561 OUT557

×

SelectEvolutionPredictionToolBFCL/multiple_114 (prediction.evolution): evolutionary-rate prediction, not modify_painting / calculate_probability.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN55 044 OUT633

×

SelectFinalSpeedToolBFCL/multiple_103 (calculate_final_speed): free-fall final speed, not chess/court/event distractors.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN72 070 OUT825

×

SelectFitnessToolBFCL/multiple_113 (calculate_fitness): evolutionary fitness from trait values/contributions, not chess/walmart/lawyer distractors. Percentages map to decimal contributions [0.4, 0.6].1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN38 715 OUT404

×

SelectForestGrowthToolBFCL/multiple_112 (forest_growth_forecast): forest growth forecast, not get_scientist_for_discovery. include_human_impact is required true: prompt says "including human impact" (ground truth = [true]).1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN93 517 OUT1 131

×

SelectGenotypeFrequencyToolBFCL/multiple_111 (calculate_genotype_frequency): genotype frequency, not factorial / find_flute.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN46 619 OUT421

×

SelectHeatCapacityToolBFCL/multiple_108 (calc_heat_capacity): gas heat capacity, not discounted_cash_flow.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN55 030 OUT770

×

SelectLightTravelTimeToolBFCL/multiple_120 (light_travel_time): light travel time, not grocery/hotel/exchange distractors.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN47 651 OUT347

×

SelectMagneticFieldToolBFCL/multiple_105 (calculate_magnetic_field): magnetic field of a loop, not concert/lawsuit distractors.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN63 405 OUT1 099

×

SelectMutationTypeToolBFCL/multiple_110 (mutation_type.find): SNP mutation type, not get_collectables_in_season.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN46 558 OUT442

Tool Calling - Tool Selection: Sports & Card Games 1 passed9 failed

10.0%

IN529 958 OUT7 266

×

SelectChessTopPlayersToolBFCL/multiple_177 (board_game.chess.get_top_players): top chess players by location/rating.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN65 116 OUT871

×

SelectDefenseRankingToolBFCL/multiple_173 (get_defense_ranking): NBA best-defence ranking by season.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN27 335 OUT337

×

SelectFindAthleteToolBFCL/multiple_172 (sports_db.find_athlete): athlete profile by name/sport.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN49 397 OUT1 509

×

SelectFindCardToolBFCL/multiple_178 (find_card_in_deck): find a card by rank/suit in default deck.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN36 227 OUT811

×

SelectGameWinnerToolBFCL/multiple_171 (game_result.get_winner): winner of a game between two teams on a date.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN65 233 OUT688

✓

SelectPokerFullHouseToolBFCL/multiple_179 (poker_probability.full_house): probability of a full house (default deck/hand).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 411 OUT119

×

SelectSoccerPlayerStatsToolBFCL/multiple_170 (soccer_stat.get_player_stats): player stats by name/season.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN49 702 OUT734

×

SelectSportsRankingToolBFCL/multiple_174 (sports_ranking): team ranking in a league.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN34 465 OUT376

×

SelectTeamScheduleToolBFCL/multiple_176 (sports_team.get_schedule): team schedule for N games in a league.1 AI query 13 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 13

Failed

IN124 402 OUT1 213

×

SelectTopPlayerToolBFCL/multiple_175 (sports_ranking.get_top_player): top player by sport/gender.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN64 670 OUT608

Tool Calling - Tool Selection: Statistics, Finance & Health 2 passed4 failed

33.3%

IN590 211 OUT8 254

✓

SelectBmiMetersToolBFCL/multiple_117 (calculate_BMI): BMI in kg/m, not soccer / hotel_booking distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN9 371 OUT208

×

SelectGcdToolBFCL/multiple_101 (math.gcd): greatest common divisor, not get_top_cases.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN35 038 OUT352

×

SelectHotelPricingToolBFCL/multiple_94 (hotel_room_pricing.get): hotel room pricing, not car/flight pricing.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN440 663 OUT4 697

×

SelectReturnOnEquityToolBFCL/multiple_128 (calculate_return_on_equity): ROE from income/equity/dividends, not library/lyrics/law distractors. dividends_paid is required 200000: prompt explicitly supplies "dividends paid of $200,000".1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN53 985 OUT963

✓

SelectSingleProbabilityToolBFCL/multiple_124 (probabilities.calculate_single): single-event probability, not math.power / fetch_DNA_sequence.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 357 OUT167

×

SelectTwoSampleTtestToolBFCL/multiple_125 (run_two_sample_ttest): two-group t-test, not restaurant/personality distractors. equal_variance is required true: prompt says "assuming equal variance" (ground truth = [true, ""]).1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN41 797 OUT1 867

Tool Calling - Tool Selection: Statistics, Finance & Services 8 failed

0.0%

IN1 003 652 OUT12 356

×

SelectBmiMetricToolBFCL/multiple_116 (calculate_bmi): BMI in kg/cm with unit option, not celebrity_net_worth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN19 215 OUT162

×

SelectCalculateProbabilityToolBFCL/multiple_123 (calculate_probability): probability from favorable/total, not lawsuit/house-price distractors.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN17 233 OUT311

×

SelectCompoundInterestToolBFCL/multiple_129 (compound_interest): compound interest, not lawsuits_search.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN29 675 OUT682

×

SelectFindRestaurantsToolBFCL/multiple_115 (find_restaurants): restaurant search with count + dietary, not instrument/stock/sports.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN520 064 OUT6 035

×

SelectLinearRegressionToolBFCL/multiple_122 (run_linear_regression): regression with predictors, not recipe/itinerary/cooking distractors. standardize is required true: prompt says "Also apply standardization" (ground truth = [true]).1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN36 221 OUT434

×

SelectMoviesByActorToolBFCL/multiple_118 (imdb.find_movies_by_actor): movies by actor+year, not flight.book / lawsuits_search.1 AI query 36 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 36

Failed

IN315 397 OUT3 375

×

SelectQuarterlyDividendToolBFCL/multiple_127 (finance.calculate_quarterly_dividend_per_share): dividend per share, not lyrics/paint/movie distractors.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN36 109 OUT502

×

SelectTTestToolBFCL/multiple_126 (t_test): two-dataset t-test, not event_finder / triangle-area distractors.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN29 738 OUT855

Tool Calling - Tool Selection: Travel & Biology 8 failed

0.0%

IN989 302 OUT9 831

×

SelectBookHotelToolBFCL/multiple_190 (book_hotel): book a room by hotel/location/room_type/start/nights.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN18 653 OUT283

×

SelectCellProteinsToolBFCL/multiple_196 (cellbio.get_proteins): proteins in a cell compartment.1 AI query 21 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 21

Failed

IN166 899 OUT1 645

×

SelectDistanceDurationToolBFCL/multiple_193 (maps.get_distance_duration): distance/duration between two places.1 AI query 15 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN114 541 OUT1 179

×

SelectForestGrowthToolBFCL/multiple_199 (forest_growth_forecast): forest growth forecast with human impact.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN60 487 OUT962

×

SelectGenotypeFrequencyToolBFCL/multiple_198 (calculate_genotype_frequency): genotype frequency from allele frequency.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN25 002 OUT304

×

SelectHeatCapacityToolBFCL/multiple_195 (calc_heat_capacity): heat capacity for a gas by temp/volume. Tightened per methodology: prompt explicitly states the gas is "air", so gas is required exactly "air" (ground truth also accepts omission, which equals the schema default).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN28 965 OUT554

×

SelectMuseumHoursToolBFCL/multiple_194 (get_museum_hours): museum opening hours on a day.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN500 520 OUT4 207

×

SelectMutationTypeToolBFCL/multiple_197 (mutation_type.find): gene mutation type by SNP id.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN74 235 OUT697

Tool Calling - Tool Selection: Triangle Properties 1 failed

0.0%

IN15 844 OUT662

×

SelectTrianglePropertiesToolBFCL/multiple_0 (triangle_properties.get): retrieves triangle dimensions/properties from 3 sides.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN15 844 OUT662

Qwen 3.6 35B NVFP4 Instruct | Benchmark #27

Tool Calling Benchmark based on BFCL

Results by difficulty

Results by category