Ai4Users - Qwen 3.6 35B NVFP4 Thinking

Benchmark items
Tool Calling - Irrelevance: Architecture & Artwork 7 passed2 failed	77.8%	IN38 099 OUT7 570
✓	NoToolForApartmentRentCostBFCL/irrelevance_158: apartment rental cost asked; only a new-construction cost estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 461 OUT382
×	NoToolForBestJazzSingerBFCL/irrelevance_164: best Jazz singer asked; only a top-instrument-players search (singers are not instrument players) is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 933 OUT1 822
✓	NoToolForCathedralCeilingDesignBFCL/irrelevance_157: how to design a cathedral ceiling asked; only a lookup of facts about an existing building is available (retrieves data, does not advise on design) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 457 OUT687
✓	NoToolForHolyBookDefinitionBFCL/irrelevance_144: definition of a holy book asked; only a search-within-a-named-holy-book tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 455 OUT448
✓	NoToolForLectureTeacherBFCL/irrelevance_167: teacher of upcoming lectures asked; only an upcoming-concert details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 458 OUT313
✓	NoToolForMostFamousSculptureBFCL/irrelevance_153: most famous sculpture asked; only a sculpting-tools lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 463 OUT892
✓	NoToolForPianoMaintenanceTipsBFCL/irrelevance_166: piano maintenance tips asked; only an instrument rental-prices lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 458 OUT951
×	NoToolForScreamArtistBFCL/irrelevance_159: artist behind 'The Scream' asked. NOTE: the only tool (artwork_search) requires a museum_location that the user did not provide, so it cannot be correctly called; BFCL labels this irrelevant. Kept faithfully with ground truth = no tool call (flagged as borderline).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 951 OUT995
✓	NoToolForSculptureRockTypeBFCL/irrelevance_152: which rock suits a garden sculpture asked; only a sculpture 3D-model creator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 463 OUT1 080
Tool Calling - Irrelevance: Buildings, Sculpture & Music 7 passed1 skipped	100.0%	IN22 989 OUT7 683
–	NoToolForCelloInstrumentTypeBFCL/irrelevance_165: instrument classification of a cello asked. NOTE: BFCL labels this irrelevant, but the only tool (get_instrument_info with detail=type) arguably answers it; flagged for review as a borderline BFCL item. Kept faithfully with ground truth = no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_instrument_info' with detail='type' answers 'what type of instrument is a cello' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForDeclarationSignersBFCL/irrelevance_137: who signed the declaration asked; only a historical-event DATE lookup is available (returns a date, not the signatories) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 282 OUT673
✓	NoToolForEiffelTowerHeightBFCL/irrelevance_156: Eiffel Tower height asked; only an architecture-plan generator (creates a new design, does not retrieve facts) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 286 OUT582
✓	NoToolForFamousDancersBFCL/irrelevance_162: famous dancers asked; only a museum-visual-artists lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 288 OUT1 491
✓	NoToolForReformationInitiatorBFCL/irrelevance_145: who initiated the Reformation asked; only a religious-event YEAR lookup is available (returns a year, not a person) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 281 OUT905
✓	NoToolForScientistDefinitionBFCL/irrelevance_143: definition of "scientist" asked; only a named-historical-figure info lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 278 OUT695
✓	NoToolForSellingGuitarBFCL/irrelevance_163: how to sell a guitar asked; only an instrument-tuning tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 283 OUT814
✓	NoToolForSimilarBlueColorBFCL/irrelevance_148: a color similar to blue asked; only a complementary-color tool (opposite, not similar) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 291 OUT2 523
Tool Calling - Irrelevance: Cell Biology & Genetics 7 passed1 failed2 skipped	87.5%	IN34 134 OUT6 603
×	NoToolForClottingBloodCellsBFCL/irrelevance_32: blood cell type responsible for clotting asked; only a nucleus-count-based cell-type classifier is available.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN7 609 OUT1 200
✓	NoToolForCommonAncestorBFCL/irrelevance_41: common ancestor of two animals asked (phylogenetics knowledge); only a fibonacci-sequence calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 789 OUT358
✓	NoToolForEvolutionaryAlgorithmInGameAIBFCL/irrelevance_43: how to apply an Evolutionary Algorithm in game AI asked (advice); only a creature-evolution simulator is available (wrong context).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 790 OUT1 210
✓	NoToolForGeneVariantFrequencyBFCL/irrelevance_36: gene-variant frequency in a population asked; only a dominant-trait-from-alleles tool is available (the true-match variant-frequency tool is deliberately isolated in A4).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 795 OUT802
–	NoToolForIdentifyingGeneticSequenceBFCL/irrelevance_33: interpret a genetic code sequence asked; only a species-identification-from-sequence tool is available (returns a species, not what the sequence means).0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'identify_species' (identifies a species from a genetic sequence) plausibly answers the query about a DNA sequence, so a competent model may call it - BFCL-mislabeled/borderline-relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForIncreasingDeerPopulationBFCL/irrelevance_39: how-to advice on growing an animal population asked; only a population-growth CALCULATOR (needs numeric rates) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 789 OUT869
✓	NoToolForLionMatingProcessBFCL/irrelevance_35: mating process of a species asked; only a dominant/recessive genetic-traits lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 786 OUT672
✓	NoToolForMitochondriaCountBFCL/irrelevance_29: number/count of mitochondria asked; only a cell-part FUNCTION lookup is available (wrong query direction).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 790 OUT863
✓	NoToolForPandaEvolutionaryHistoryBFCL/irrelevance_42: evolutionary history asked (a narrative fact); only a biodiversity-index calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 786 OUT629
–	NoToolForWhaleGeneSequenceBFCL/irrelevance_44: the actual gene sequence of a species asked; only a gene-sequence GENERATOR (needs numeric mutation rate + duration) is available.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'gene_sequencer' (generate gene sequences for evolutionary changes) directly answers the query - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
Tool Calling - Irrelevance: Color, Geometry & Population 11 passed	100.0%	IN40 836 OUT6 454
✓	NoToolForColorInComputerVisionBFCL/irrelevance_131: color meaning in computer vision asked; only a color-in-personality-psychology analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 712 OUT873
✓	NoToolForDeclarationSigningDateBFCL/irrelevance_135: signing date asked; only date-arithmetic (add days) and age tools are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 710 OUT427
✓	NoToolForElectricityDiscovererBFCL/irrelevance_140: who discovered electricity asked; only an electric-current calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 706 OUT627
✓	NoToolForHexColorMixBFCL/irrelevance_146: mixing two hex colors asked; only a prophet-details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 722 OUT812
✓	NoToolForIndependenceSigningDateBFCL/irrelevance_138: signing date asked; only an age calculator (needs birthdate + current date) and a date-adder are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 710 OUT433
✓	NoToolForMostVisitedMarketBFCL/irrelevance_161: most visited market asked; only a museum visitation-statistics lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 712 OUT714
✓	NoToolForNbaFinalWinnerBFCL/irrelevance_133: NBA final winner asked; only a historical-battle details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 713 OUT227
✓	NoToolForPicassoPaintingYearBFCL/irrelevance_151: year of a Picasso painting asked; only a sculpture-dimensions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 711 OUT556
✓	NoToolForRelativityAuthorBFCL/irrelevance_142: who proposed special relativity asked; only a chemical-element properties lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 715 OUT248
✓	NoToolForSculptureCreatorBFCL/irrelevance_155: sculpture creator asked; only a max-seating-capacity calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 711 OUT910
✓	NoToolForWhichColorsToMixBFCL/irrelevance_150: which colors to mix for a shade asked; only an RGB-to-Pantone converter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 714 OUT627
Tool Calling - Irrelevance: Cooking, Currency & Markets 6 passed2 failed	75.0%	IN32 473 OUT8 295
×	NoToolForBakerySourdoughBFCL/irrelevance_223: a BAKERY selling sourdough asked; only a grocery-shop product finder available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 506 OUT1 582
✓	NoToolForChickenRecipeBFCL/irrelevance_211: the RECIPE to cook chicken asked; only a cook-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 241 OUT834
✓	NoToolForChocolateCakeIngredientsBFCL/irrelevance_220: cake INGREDIENTS asked; only a cooking-TEMPERATURE tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 244 OUT1 044
✓	NoToolForCurrencyToGbpBFCL/irrelevance_233: CURRENCY conversion asked; only a WEATHER tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 246 OUT440
×	NoToolForHowToBoilAnEggBFCL/irrelevance_212: best WAY to boil an egg asked; only a boiling-TIME calculator available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 512 OUT2 066
✓	NoToolForLasagnaRecipeBFCL/irrelevance_219: the RECIPE for lasagna asked; only a recipe-CALORIE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 241 OUT1 232
✓	NoToolForLegExercisesBFCL/irrelevance_221: leg EXERCISES asked; only a grocery-list generator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 241 OUT475
✓	NoToolForPetStoreBFCL/irrelevance_224: a PET store asked; only a GROCERY-store locator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 242 OUT622
Tool Calling - Irrelevance: Dates & Elements 9 passed1 failed	90.0%	IN39 945 OUT9 546
✓	NoToolForHydrogenPropertiesBFCL/irrelevance_141: chemical properties of Hydrogen asked; only a scientist-contributions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 626 OUT1 560
✓	NoToolForLargestPlanetBFCL/irrelevance_139: largest planet asked; only a star-info lookup (needs a star name) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 626 OUT1 176
✓	NoToolForMostImportantProphetBFCL/irrelevance_147: most important prophet asked; only a two-color mixer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 626 OUT363
×	NoToolForMuseumVisitFrequencyBFCL/irrelevance_160: museum member visit frequency asked; only a single most-frequent-visitor lookup is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN7 306 OUT3 001
✓	NoToolForPantoneOfSkyBlueBFCL/irrelevance_149: Pantone code of sky blue asked; only a paint-proportions calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 628 OUT1 197
✓	NoToolForPersonalityTypeBFCL/irrelevance_130: user asks what personality type they are; only a big-five calculator (which needs the user's rated answers) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 624 OUT493
✓	NoToolForStadiumSeatingCapacityBFCL/irrelevance_154: stadium seating capacity asked; only a sculpture-creator lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 627 OUT470
✓	NoToolForVicePresidentBFCL/irrelevance_136: Vice President asked; only a US-president-by-year lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 626 OUT408
✓	NoToolForWaterlooCasualtiesBFCL/irrelevance_132: casualty count of a battle asked; only a historical-event date lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 628 OUT685
✓	NoToolForWorldCupWinnerBFCL/irrelevance_134: sports World Cup winner asked; only a historical-battle outcome predictor is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 628 OUT193
Tool Calling - Irrelevance: Electromagnetism, Thermo & Data 10 passed	100.0%	IN34 045 OUT6 124
✓	NoToolForElectronRestMassBFCL/irrelevance_23: mass of an electron asked (a knowledge fact); only magnetic-field / current tools are available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 397 OUT179
✓	NoToolForFlightDurationBFCL/irrelevance_20: flight duration asked; only a currency converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 400 OUT323
✓	NoToolForFreezingPointBFCL/irrelevance_25: freezing point asked; only boiling-point calculators are available (wrong phase transition).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 407 OUT830
✓	NoToolForGasAmountGeneratedBFCL/irrelevance_26: amount of gas generated asked; only a gas-PRESSURE calculator is available (wrong quantity).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 420 OUT1 122
✓	NoToolForGiniCoefficientBFCL/irrelevance_48: Gini coefficient asked; only a GDP-fetch tool is available (different economic indicator).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 400 OUT498
✓	NoToolForHeatingWaterBFCL/irrelevance_27: heat energy to warm water asked. NOTE: the original tool (calculate_heat) is a fair match for this query (only the well-known specific heat is missing), which would make "no call" ambiguous; per human-baseline fairness it is deliberately NOT shown here (kept as a distractor in A3), so no visible tool fits and "no call" is unambiguously correct.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 411 OUT1 948
✓	NoToolForHexagonSidesBFCL/irrelevance_28: number of sides of a hexagon asked (basic geometry knowledge); only a boiling-point tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 398 OUT179
✓	NoToolForSymbolicMagneticFieldBFCL/irrelevance_22: magnetic field asked with SYMBOLIC values (distance 'r', current 'I'); the two magnetic-field tools require concrete numeric distance + current, so there is nothing to pass -> no fit.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 411 OUT457
✓	NoToolForTimeDifferenceBFCL/irrelevance_49: time-zone difference asked; only a geo-coordinate lookup tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 400 OUT384
✓	NoToolForWorldSeriesWinnerBFCL/irrelevance_53: sports trivia asked; only a SQL database-query runner is available (no database/query provided).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 401 OUT204
Tool Calling - Irrelevance: Entertainment, Sports & Knowledge 9 passed	100.0%	IN31 410 OUT3 257
✓	NoToolForBasketballResultBFCL/irrelevance_84: basketball game result asked; only a stock-price retrieval tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 490 OUT272
✓	NoToolForBestSoccerPlayerBFCL/irrelevance_79: best current soccer player asked; only a stock-market prediction tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 488 OUT970
✓	NoToolForCaliforniaGovernorBFCL/irrelevance_87: current Governor of California asked; only a court-case filing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 486 OUT239
✓	NoToolForCrimeThrillerMovieListBFCL/irrelevance_88: best crime-thriller movies of 2020 asked; only a document forgery detector is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 495 OUT266
✓	NoToolForFifaWorldCupWinnerBFCL/irrelevance_80: FIFA World Cup 2010 winner asked; only a stock-ticker lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 491 OUT174
✓	NoToolForFrenchTranslationBFCL/irrelevance_61: French translation of a word asked; only an SVM-model tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 490 OUT216
✓	NoToolForHorrorMovieListBFCL/irrelevance_77: horror movie recommendations asked; only an investment future-value tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 489 OUT738
✓	NoToolForPresidentialElectionWinnerBFCL/irrelevance_85: 2020 presidential election winner asked; only a criminal-case detail lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 491 OUT181
✓	NoToolForSportsNewsBFCL/irrelevance_76: latest sports news asked; only a portfolio investment-trend analyzer is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 490 OUT201
Tool Calling - Irrelevance: Finance & Economics 7 passed1 failed	87.5%	IN30 189 OUT4 164
✓	NoToolForAverageHousePriceBFCL/irrelevance_69: market-wide average house price asked; only a regression predictor (needs a trained model + feature vector, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 346 OUT345
✓	NoToolForCompoundInterestBFCL/irrelevance_70: compound-interest computation asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 372 OUT976
✓	NoToolForGoldPriceBFCL/irrelevance_78: today's gold spot price asked; only a bond-price calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 348 OUT199
✓	NoToolForHorrorMovieCatalogBFCL/irrelevance_81: horror movie recommendations asked; only a historical stock-price fetcher is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 349 OUT434
×	NoToolForLossProjectionBFCL/irrelevance_73: a loss projection asked; only a revenue-prediction tool is available (revenue != loss).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 714 OUT1 136
✓	NoToolForPoundToKilogramBFCL/irrelevance_75: a mass unit conversion asked; only a portfolio-returns calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 347 OUT167
✓	NoToolForProfitMarginBFCL/irrelevance_71: profit margin asked; only an ROI calculator is available (ROI != margin) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 369 OUT556
✓	NoToolForStockNewsBFCL/irrelevance_82: stock news asked; only a capital-gains calculator is available (gains != news) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 344 OUT351
Tool Calling - Irrelevance: Finance Near-Matches 8 passed1 failed	88.9%	IN34 411 OUT12 102
✓	NoToolForBitcoinPriceBFCL/irrelevance_56: current bitcoin price asked; only a generic database record-select tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 426 OUT772
✓	NoToolForBookstoreLookupBFCL/irrelevance_55: which online bookstore sells a title asked; only an add-to-cart tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 431 OUT470
✓	NoToolForBusinessRateOfReturnBFCL/irrelevance_74: rate of return asked; only a net-profit calculator is available (profit != rate of return).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 447 OUT1 173
✓	NoToolForCoinFlipProbabilityBFCL/irrelevance_66: probability of a coin landing heads asked; only a standard-deviation tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 431 OUT434
✓	NoToolForCurrentInterestRateBFCL/irrelevance_83: the current market interest rate asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 425 OUT2 816
×	NoToolForExternalRateOfReturnBFCL/irrelevance_72: EXTERNAL rate of return asked; only an internal-rate-of-return (IRR) tool is available (ERR != IRR, even though cash flows are given) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 945 OUT4 236
✓	NoToolForFinalPriceWithoutBasePriceBFCL/irrelevance_60: final price after discount+tax asked, but the original price is never given, so the (domain-matching) calculateFinalPrice cannot be correctly called -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 445 OUT280
✓	NoToolForFrontendDevelopmentBFCL/irrelevance_62: how to build an e-commerce frontend asked; only a recommender-model builder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 431 OUT486
✓	NoToolForHighestGrossingMovieBFCL/irrelevance_54: highest-grossing movie of all time asked; only a movie search (requires a specific title + year, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 430 OUT1 435
Tool Calling - Irrelevance: Food, Time & Distance 2 passed2 failed1 skipped	50.0%	IN16 296 OUT5 543
×	NoToolForCakeBakingMachineBFCL/irrelevance_218: a baking MACHINE/appliance asked; only a recipe retriever available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN5 450 OUT2 462
✓	NoToolForDrivingTravelTimeBFCL/irrelevance_215: driving TRAVEL TIME asked; only a restaurant-bill tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 710 OUT781
×	NoToolForLondonTimezoneNameBFCL/irrelevance_226: the NAME of London's timezone asked; only a local-TIME-by-timezone tool available (it needs the timezone as input and returns a time, not the timezone name) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN5 422 OUT1 213
–	NoToolForPizzaInBostonBFCL/irrelevance_213: a place for pizza asked; a cuisine+location restaurant_finder is available. REVIEW FLAG (fairness): restaurant_finder(cuisine=pizza/Italian, location=Boston) plausibly answers this, so a competent human might legitimately call it. BFCL labels it irrelevant; kept as such and marked Hard because the intended "no call" is debatable. No OTHER query here is a restaurant search, so the tool cannot wrongly satisfy the file's other cases.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'restaurant_finder' (cuisine+location) answers 'good place for pizza in Boston' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForTriangleHypotenuseBFCL/irrelevance_238: the HYPOTENUSE of a triangle asked; only a geo lat/long distance calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 714 OUT1 087
Tool Calling - Irrelevance: Geo, Travel & Sentiment 7 passed1 failed1 skipped	87.5%	IN30 713 OUT5 087
✓	NoToolForAverageTemperatureBmiBFCL/irrelevance_110: an average December temperature asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 396 OUT558
✓	NoToolForCaseLawyerBFCL/irrelevance_95: the lawyer of a case asked; only a case-outcome retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 399 OUT752
✓	NoToolForCurrentTimeSunriseBFCL/irrelevance_102: the current time asked; only a sunrise/sunset time calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 395 OUT299
✓	NoToolForFamousQuoteBFCL/irrelevance_119: a famous quote asked; only a sentiment-analysis tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 393 OUT138
✓	NoToolForFastestRouteBFCL/irrelevance_114: the fastest route asked; only a closest-airport finder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 397 OUT574
–	NoToolForRestaurantsInLondonBFCL/irrelevance_113: restaurants in London asked; only a generic points-of-interest locator is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'find_pois' (locate points of interest by category) answers 'restaurants in London' (restaurants are POIs) - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForSportsBrainImpactBFCL/irrelevance_120: the neurological impact of sports asked; only a caffeine neurological-impact tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 396 OUT688
✓	NoToolForTrafficConditionBFCL/irrelevance_112: the current traffic condition asked; only an address geocoder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 395 OUT274
×	NoToolForTravelTimeByCarBFCL/irrelevance_115: travel time by car asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2	Failed	IN6 942 OUT1 804
Tool Calling - Irrelevance: Legal & Case Lookup 9 passed1 failed	90.0%	IN39 074 OUT4 620
✓	NoToolForAppleCaloriesBFCL/irrelevance_100: calories in an apple asked; only a litigation-cost calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 548 OUT176
✓	NoToolForBrainWeightBFCL/irrelevance_122: average human brain weight asked; only a per-patient neural-activity tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 550 OUT232
✓	NoToolForCaseJudgementBFCL/irrelevance_93: a case judgement asked; only a law-firm impactful-cases lister is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 548 OUT396
✓	NoToolForChineseRestaurantBFCL/irrelevance_97: a Chinese-restaurant recommendation asked; only a lawsuit-filing tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 551 OUT340
✓	NoToolForMovieRecommendationBFCL/irrelevance_116: a movie recommendation asked; only a word-count tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 549 OUT737
×	NoToolForNbaChampionBFCL/irrelevance_118: the 1996 NBA champion asked; only an NBA playoff-brackets display tool is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN7 118 OUT955
✓	NoToolForPaintingEiffelSettlementBFCL/irrelevance_98: how long to paint the Eiffel Tower asked; only a lawsuit-settlement estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 552 OUT283
✓	NoToolForSecurityGuardSearchBFCL/irrelevance_92: finding a security guard asked; only a lawyer-search tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 558 OUT418
✓	NoToolForSphereVolumeBFCL/irrelevance_104: sphere volume asked; only a circular-park area tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 552 OUT752
✓	NoToolForTrafficLawsBFCL/irrelevance_99: traffic laws asked; only a lawsuit-search tool is available (lawsuits, not laws) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 548 OUT331
Tool Calling - Irrelevance: Math & Science Distractors 8 passed	100.0%	IN25 236 OUT5 547
✓	NoToolForDefiniteIntegralBFCL/irrelevance_9: definite integral asked; only an income-tax calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 157 OUT460
✓	NoToolForIntegralOfCubeBFCL/irrelevance_8: definite integral asked; only a string-to-int converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 150 OUT1 212
✓	NoToolForLatestMoviesBFCL/irrelevance_15: latest movie releases asked; only a physics final-velocity tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 144 OUT247
✓	NoToolForPerpendicularSlopeBFCL/irrelevance_3: slope of a perpendicular line asked; only a critical-points finder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 162 OUT1 155
✓	NoToolForPrimeFactorsBFCL/irrelevance_13: prime factorization asked; only a compound-interest calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 147 OUT341
✓	NoToolForRectangleAreaBFCL/irrelevance_6: rectangle area asked; only a circle-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 158 OUT227
✓	NoToolForTriangleAreaBFCL/irrelevance_0: triangle area asked; only a BMI tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 158 OUT398
✓	NoToolForUpwardAccelerationBFCL/irrelevance_14: acceleration asked (it is just gravity); the tempting max-height and final-velocity tools compute neither the acceleration nor what is asked -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 160 OUT1 507
Tool Calling - Irrelevance: Math & Statistics 5 passed1 failed	83.3%	IN21 155 OUT7 989
✓	NoToolForBinomialMeanBFCL/irrelevance_67: the MEAN of a binomial experiment asked; only p-value tools are available (they return a p-value, not the mean) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 026 OUT2 627
✓	NoToolForCivilLawComponentsBFCL/irrelevance_90: the components/categories of Civil Law asked; only a noise-complaint filing tool is available (filing a complaint does not explain law) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN2 998 OUT661
✓	NoToolForCoinTossMaxHeadsBFCL/irrelevance_63: how many heads CAN be obtained from tossing 3 coins (a count/maximum, not a probability) asked; only a probability-of-an-event calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 002 OUT366
×	NoToolForNullHypothesisStdDevBFCL/irrelevance_68: the STANDARD DEVIATION of a null-hypothesis test asked; only p-value tools are available (they return a p-value, not a standard deviation) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 121 OUT1 955
✓	NoToolForPowerOfTwoBFCL/irrelevance_59: raising a number to a power asked; only a linear-equation solver is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 001 OUT1 356
✓	NoToolForQuadraticEquationBFCL/irrelevance_57: solving a quadratic equation asked; only a genetic-algorithm optimizer and a LINEAR equation solver are available (a quadratic is not linear) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 007 OUT1 024
Tool Calling - Irrelevance: Medical, Social & Personality 6 passed3 skipped	100.0%	IN20 452 OUT4 664
✓	NoToolForBigMacCaloriesBFCL/irrelevance_123: calories of a Big Mac asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 407 OUT461
✓	NoToolForEconomicHappinessEffectBFCL/irrelevance_126: the effect of economic status on happiness asked; only a happiness-index lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 409 OUT2 239
✓	NoToolForFriendMbtiBFCL/irrelevance_129: a friend's MBTI asked; only a personality-type evaluator (needs trait percentages) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 407 OUT269
–	NoToolForLatestTechTrendBFCL/irrelevance_124: the latest technology trend asked; only a trending-topics-by-category tool is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_social_trends' (trending topics by category) answers 'latest trend in technology' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForMajorAirportsBFCL/irrelevance_117: major US airports asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 410 OUT701
–	NoToolForMotorNeuronDiseaseBFCL/irrelevance_121: motor neuron disease info asked; only a generic disease-info retriever is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'medical_records_get_disease_info' directly answers 'information on motor neuron diseases' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForPopularBooksBFCL/irrelevance_125: popular books by an author asked; only a recent-tweets retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 409 OUT458
–	NoToolForTwitterMoodBFCL/irrelevance_127: overall twitter mood asked; only a twitter sentiment-analysis tool is available -> BFCL ground truth is no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'sentiment_analysis_twitter' directly answers 'general mood of twitter regarding the new iPhone' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForVegetableServingsBFCL/irrelevance_128: daily vegetable servings asked; only a personality-test scorer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 410 OUT536
Tool Calling - Irrelevance: Mixed Distractors 8 passed	100.0%	IN25 992 OUT4 608
✓	NoToolForAirQualityBFCL/irrelevance_40: current air quality asked; only a plant-biomass calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 250 OUT330
✓	NoToolForFavoriteRestaurantBFCL/irrelevance_31: a favorite restaurant recommendation asked; only a cell-division simulator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 246 OUT300
✓	NoToolForGlobalWarmingDocumentaryBFCL/irrelevance_38: a documentary (video) asked; only a scientific-research-paper retriever is available (wrong medium).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 247 OUT965
✓	NoToolForLasagnaIngredientsBFCL/irrelevance_47: recipe ingredients asked; only a flight-timetable tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 247 OUT495
✓	NoToolForLionDominantTraitBFCL/irrelevance_34: dominant genetic trait of a species asked; only a gene-variant-FREQUENCY lookup is available (different genetic output).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 249 OUT1 155
✓	NoToolForMultinucleatedCellNameBFCL/irrelevance_30: name of a cell type with multiple nuclei asked; only a blood-cell classifier (by shape/size) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 253 OUT478
✓	NoToolForPicnicSpotBFCL/irrelevance_37: a picnic spot asked; only a local-fauna information tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 246 OUT654
✓	NoToolForTranslationBFCL/irrelevance_52: natural-language translation asked; only a file-compression tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 254 OUT231
Tool Calling - Irrelevance: Motion, Electricity & Misc 10 passed	100.0%	IN35 408 OUT6 308
✓	NoToolForCarProjectileRangeBFCL/irrelevance_16: projectile-style "how far will a car travel" asked with symbolic v/t/theta; the range tool needs concrete numeric velocity + angle -> nothing to pass -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 551 OUT696
✓	NoToolForConeVolumeBFCL/irrelevance_19: cone volume asked; only an investment-return calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 545 OUT650
✓	NoToolForCurrentClockTimeBFCL/irrelevance_17: current clock time asked; only a travel-time (distance/speed) calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 534 OUT207
✓	NoToolForElectronMassBFCL/irrelevance_24: mass of an electron asked (a knowledge fact); only an electric-current tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 535 OUT302
✓	NoToolForFibonacciOf20BFCL/irrelevance_51: fibonacci number asked; only a cryptocurrency-price tool is available (the fibonacci calculator lives in a different file so it cannot satisfy this query).1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 536 OUT421
✓	NoToolForForceAngleFromMomentumBFCL/irrelevance_18: angle of a force from momentum asked; only a vector-angle-from-XY-components tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 541 OUT1 299
✓	NoToolForHorrorMovieListBFCL/irrelevance_50: movie recommendations asked; only a Celsius-to-Fahrenheit converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 541 OUT742
✓	NoToolForMagneticFieldNearWireBFCL/irrelevance_21: magnetic field near a current-carrying wire asked; only an EM-wave-amplitude tool is available (magnetic-field calculators are deliberately kept in a different file).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 550 OUT1 194
✓	NoToolForSineOf45DegreesBFCL/irrelevance_45: sine of an angle asked; only a polygon-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 536 OUT364
✓	NoToolForTeslaPriceBFCL/irrelevance_46: product price of a car asked; only a currency-exchange-rate tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 539 OUT433
Tool Calling - Irrelevance: Music Theory & Lookup 8 passed	100.0%	IN25 572 OUT4 451
✓	NoToolForAncientGreekLawmakersBFCL/irrelevance_171: who established laws in Ancient Greece asked (history); only a musical-piece composer search is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 194 OUT558
✓	NoToolForBasketballGameAtVenueBFCL/irrelevance_169: who is playing a basketball game at a venue asked; only concert-locating tools exist.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 197 OUT360
✓	NoToolForBluesScalePatternBFCL/irrelevance_176: the interval pattern of a blues scale asked; only composer lookups are available (get_blues_scale lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 199 OUT832
✓	NoToolForDonQuixoteAuthorBFCL/irrelevance_172: author of the novel Don Quixote asked; only music-composition lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 192 OUT220
✓	NoToolForFamousUsComposersBFCL/irrelevance_170: most famous US composers asked (general knowledge, no specific piece); only a chord-progression creator + piece-specific composer lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 195 OUT1 002
✓	NoToolForGrammyBestAlbumBFCL/irrelevance_177: Grammy Award winner asked; only a song chord-progression lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 200 OUT606
✓	NoToolForPrimaryTriadsOfKeyBFCL/irrelevance_173: primary triads of a key asked; only a genre-common-chords finder is available (primary_triads lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 198 OUT571
✓	NoToolForUniversityClassAvailabilityBFCL/irrelevance_168: availability of a university class asked; only concert-availability / concert-search tools are available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 197 OUT302
Tool Calling - Irrelevance: Probability & Legal 5 passed	100.0%	IN13 997 OUT5 720
✓	NoToolForFaceCardProbabilityBFCL/irrelevance_64: probability of drawing a face card from a deck asked; only a COIN-toss heads probability tool is available (coins, not cards) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 796 OUT469
✓	NoToolForHomeElectricityNeedBFCL/irrelevance_58: home electricity demand (kWh) asked; only a solar-panel COUNT calculator is available (it sizes panels, not electricity need, and needs sunlight hours not given) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN2 799 OUT1 158
✓	NoToolForNoiseComplaintProcessBFCL/irrelevance_91: how/whether to report a noise complaint asked; only a law-category listing tool is available (it lists legal categories, it does not file or advise on complaints) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 796 OUT782
✓	NoToolForRedMarbleCountBFCL/irrelevance_65: number of red marbles asked; only a determine-population tool is available (it returns a total population from probability + sample size, not a sub-category count) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN2 811 OUT1 591
✓	NoToolForTenantRightsBFCL/irrelevance_89: explaining tenant rights asked; only a contract-generation tool is available (generating a contract does not explain rights) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN2 795 OUT1 720
Tool Calling - Irrelevance: Recipes & Cooking 8 passed1 skipped	100.0%	IN27 642 OUT5 867
✓	NoToolForBestGameCharacterBFCL/irrelevance_206: best game character vs a dragon asked; only a best-WEAPON finder available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 459 OUT1 853
✓	NoToolForBestMovie2020BFCL/irrelevance_235: best MOVIE of 2020 asked; only a currency-exchange calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 456 OUT338
✓	NoToolForBuildingAPcBFCL/irrelevance_209: how to build a PC asked; only a cooking-recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 451 OUT1 055
✓	NoToolForFamousPlaceInParisBFCL/irrelevance_210: most famous PLACE in Paris asked; only a restaurant-by-dish search available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 453 OUT346
✓	NoToolForGameShortestPathBFCL/irrelevance_208: shortest in-game path asked; only a taxi-fare calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 461 OUT417
✓	NoToolForInchToMeterBFCL/irrelevance_234: LENGTH conversion (inches to metres) asked; only a CURRENCY converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 452 OUT273
✓	NoToolForLowestGameScoreBFCL/irrelevance_207: LOWEST game score asked; only a HIGHEST-score retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 456 OUT801
✓	NoToolForNewYorkPopulationBFCL/irrelevance_225: current POPULATION of New York asked; only a time-zone converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 454 OUT784
–	NoToolForQuickestFlightLondonTokyoBFCL/irrelevance_236: quickest way to fly London->Tokyo asked; a flight-duration tool is available. REVIEW FLAG (fairness): get_flight_duration plausibly answers this ("quickest ... by plane" ~= quickest flight duration), so a competent human might legitimately call it. Kept as BFCL labels it (irrelevant), but this item is borderline. Marked Hard because the intended "no call" is debatable.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_flight_duration' (with flight_type non-stop) plausibly answers 'quickest way ... by plane' - BFCL-mislabeled/borderline-relevant. Kept for traceability.	Skipped	IN0 OUT0
Tool Calling - Irrelevance: Restaurants & Travel 7 passed1 failed	87.5%	IN29 855 OUT4 147
✓	NoToolForBeijingSunsetBFCL/irrelevance_227: SUNSET time asked; only a SUNRISE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 313 OUT412
✓	NoToolForClosestItalianRestaurantBFCL/irrelevance_216: FIND the closest Italian restaurant asked; only a currency converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 312 OUT236
✓	NoToolForFindSushiRestaurantBFCL/irrelevance_214: FIND a sushi restaurant asked; only a tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 314 OUT209
✓	NoToolForNearestPharmacyBFCL/irrelevance_237: FIND the nearest pharmacy asked; only a route-to-a-known-location planner available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 314 OUT355
✓	NoToolForPetFriendlyTrainStationBFCL/irrelevance_230: a pet-friendly TRAIN STATION asked; only hotel search tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 313 OUT256
✓	NoToolForSushiRestaurantsTokyoBFCL/irrelevance_229: FIND sushi RESTAURANTS asked; only hotel-booking tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 314 OUT277
✓	NoToolForThaiRestaurantChicagoBFCL/irrelevance_231: FIND a Thai RESTAURANT asked; only a hotel-availability checker available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 315 OUT570
×	NoToolForYogaRetreatBFCL/irrelevance_232: a yoga RETREAT asked; only a HOTEL search tool available. REVIEW FLAG (fairness): a retreat is not a hotel, but hotel_search_find_hotels accepts arbitrary amenities (spa, vegetarian) and a location, so it is somewhat tempting. BFCL labels this irrelevant (retreat != hotel); kept as such and marked Hard because the boundary is borderline.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 660 OUT1 832
Tool Calling - Irrelevance: Sports & Games (set 1) 9 passed1 failed	90.0%	IN39 285 OUT9 061
✓	NoToolForAssistAverageBFCL/irrelevance_185: assist average asked; only an average-SCORING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 570 OUT1 167
✓	NoToolForBestChessMoveBFCL/irrelevance_194: best next move asked; only a board-game-STATISTICS tool is available (the move-suggesting board_analyser lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 572 OUT567
✓	NoToolForChessArrangementCountBFCL/irrelevance_196: total theoretical arrangements asked; only a final-score calculator is available (possible_moves lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 572 OUT733
✓	NoToolForFootballTeamRankingBFCL/irrelevance_186: ranking of a team asked; only a season-MVP tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 568 OUT897
✓	NoToolForIceCreamStoreBFCL/irrelevance_201: find a physical store asked; only a video-game guide is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 565 OUT319
×	NoToolForMostHeartsInPokerBFCL/irrelevance_199: who holds most of a suit asked; only a poker-dealing tool is available (get_highest_card_holder lives in E4) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN7 169 OUT1 966
✓	NoToolForPlayerBiographyBFCL/irrelevance_189: a biographical "who is X" asked; only a top-ranked-athlete lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 565 OUT818
✓	NoToolForTeamRosterBFCL/irrelevance_192: full roster asked; only a top-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 567 OUT1 221
✓	NoToolForUnoRulesBFCL/irrelevance_198: the rules of a game asked; only a deck-shuffling tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 569 OUT990
✓	NoToolForWorldCupWinnerBFCL/irrelevance_191: who won a tournament asked (no team named); only a per-team match-stats tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 568 OUT383
Tool Calling - Irrelevance: Sports & Games (set 2) 8 passed1 failed	88.9%	IN34 683 OUT8 564
✓	NoToolForBlackjackAceRuleBFCL/irrelevance_200: a rules question asked; only a poker highest-card-holder tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 467 OUT490
✓	NoToolForBookAuthorBFCL/irrelevance_205: author of a book asked; only a video-game score calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 471 OUT264
✓	NoToolForChessTournamentWinnerBFCL/irrelevance_195: who won a tournament asked; only a board-position move-suggester is available (the game-history fetch_game_stats lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 468 OUT620
✓	NoToolForCreateJigsawBFCL/irrelevance_204: CREATE a puzzle asked; only a puzzle-SOLVING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 461 OUT423
✓	NoToolForHaloPlayerRankBFCL/irrelevance_203: a player's RANK asked; only a player-SCORE lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 468 OUT1 424
×	NoToolForLakersTopScorerBFCL/irrelevance_193: top scorer of a team asked; only a team-details tool (roster/results/upcoming, not scorers) is available (sports_team.get_top_scorer lives in E3) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 954 OUT2 545
✓	NoToolForMonopolyWinnerBFCL/irrelevance_197: who won a casual game asked; only a possible-moves calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 466 OUT511
✓	NoToolForTopTennisPlayerBFCL/irrelevance_190: top ranked PLAYER asked; only a TEAM standing/ranking tool is available (the athlete-ranking get_top_ranked lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 465 OUT913
✓	NoToolForWorldSeriesWinnerBFCL/irrelevance_202: who won a real game asked; only a score calculator (from raw points) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 463 OUT1 374
Tool Calling - Irrelevance: Sports Statistics 9 passed	100.0%	IN31 560 OUT9 984
✓	NoToolForBasketballSeasonMvpBFCL/irrelevance_187: who won the MVP asked; only a team-ranking tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 509 OUT554
✓	NoToolForClippersRosterBFCL/irrelevance_179: roster of a team asked; only a two-team game-results tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 502 OUT976
✓	NoToolForCommonChordsInPopBFCL/irrelevance_174: most common chords in a pop song asked; only a key-signature primary-triads tool is available (find_common_chords lives in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 507 OUT1 760
✓	NoToolForCricketMatchParticipantsBFCL/irrelevance_180: who is playing in cricket matches today asked; only a cricket-schedule tool (dates/matches, not participants) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 506 OUT1 467
✓	NoToolForLaLigaPlayersBFCL/irrelevance_181: who played in a whole league asked; only a two-named-team last-match-result tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 502 OUT958
✓	NoToolForMoonlightSonataComposerBFCL/irrelevance_175: composer of Moonlight Sonata asked; only a blues-scale generator is available here (all composer-lookup tools are isolated in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 505 OUT206
✓	NoToolForNbaMvpWinnerBFCL/irrelevance_184: who won an MVP award asked (no player named); only a per-player stats tool that requires a player name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 511 OUT577
✓	NoToolForTopAssistPlayerBFCL/irrelevance_178: top ASSIST player asked; only a top-GOAL-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 506 OUT3 123
✓	NoToolForWimbledonChampionBFCL/irrelevance_183: winner of a tournament asked (no person named); only a celebrity-info tool that requires a name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 512 OUT363
Tool Calling - Irrelevance: Time, Weather & Misc 2 passed1 failed1 skipped	66.7%	IN10 341 OUT4 264
×	NoToolForCityDistanceKmBFCL/irrelevance_239: DISTANCE (km) between two cities asked; only a get_date tool that returns the TIME difference between two locations is available (its 'unit' arg mentions distance, but it computes time) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN5 187 OUT2 524
–	NoToolForCurrentTimeSydneyBFCL/irrelevance_228: current time in Sydney asked; a local-time-by-city tool is available. REVIEW FLAG (fairness): get_local_time(location='Sydney') directly answers this, so a competent human would legitimately call it. BFCL nonetheless files it under irrelevance; kept as such to preserve the source label, but this item is effectively RELEVANT and should be reviewed. Marked Hard because the "no call" expectation is contradicted by the tool itself.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_local_time' directly answers 'current time in Sydney' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForTomatoCaloriesBFCL/irrelevance_222: CALORIES in a tomato asked; only a grocery-item-details lookup available. REVIEW FLAG (fairness): item_details at 'detailed' level could arguably surface nutrition info, and it requires a store_location the user never gave; BFCL labels it irrelevant (it is a grocery-store item lookup, not a nutrition-facts service). Kept as such; marked Medium as borderline.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 578 OUT1 207
✓	NoToolForWriteABookBFCL/irrelevance_217: "can you write a book" asked; only a recipe-CREATION tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN2 576 OUT533
Tool Calling - Irrelevance: Weather, Environment & Services 9 passed1 failed	90.0%	IN40 303 OUT11 317
✓	NoToolForBestCo2ReductionBFCL/irrelevance_108: the best way to reduce CO2 asked; only a numeric emission-reduction estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 660 OUT1 183
✓	NoToolForBestMonthToVisitBFCL/irrelevance_101: best month to visit Hawaii asked; only an average-monthly-temperature tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 658 OUT1 003
×	NoToolForCactusNutrientsBFCL/irrelevance_109: a plant's nutrient needs asked; only a weekly watering-needs calculator is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN7 351 OUT3 410
✓	NoToolForCurrentTimeWeatherBFCL/irrelevance_103: the current time asked; only a current-weather forecast tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 659 OUT936
✓	NoToolForGardenSpeciesCompositionBFCL/irrelevance_107: garden species composition asked; only a soil sand/silt/clay composition analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 662 OUT388
✓	NoToolForGmoYogaMatBFCL/irrelevance_111: a GMO yoga mat asked; only an eco-friendly product locator is available (GMO is not eco-friendly) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 661 OUT854
✓	NoToolForImpactfulCasesBFCL/irrelevance_94: a firm's impactful cases asked; only a single-case detail lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 669 OUT448
✓	NoToolForPaintingEiffelCaseResultBFCL/irrelevance_96: how long to paint the Eiffel Tower asked; only a law-case result lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 661 OUT688
✓	NoToolForPollinationFlowersBFCL/irrelevance_105: top pollination flower species asked; only a route elevation-profile plotter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 663 OUT1 281
✓	NoToolForTomatoFertilizerBFCL/irrelevance_106: best fertilizer for tomatoes asked; only a soil-type analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 659 OUT1 126
Tool Calling - Parallel Calls: Art, Music & Local Services 8 passed2 failed1 skipped	80.0%	IN82 380 OUT16 756
–	CalculateDrivingDistancesShortestRouteBFCL/parallel_38: two calc_distance calls — New York→Washington DC and Los Angeles→San Francisco. Tightened per methodology: the prompt explicitly enables "shortest route", so shortest_route == true is required (schema default is false).0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Skipped: Ambiguous scope (item flaw, not model behavior): the trailing 'with optional parameter shortest route enabled' can reasonably apply to only the second leg (its nearest clause) or to both; the BFCL ground truth requires it on both, but a competent human would not answer deterministically. Kept for traceability.	Skipped	IN0 OUT0
✓	CheckInstrumentAvailabilityInTwoCitiesBFCL/parallel_42: two check_instrument_availability calls — Yamaha P125 in Berlin and Madrid.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 486 OUT874
✓	FindMusicStoresInTwoCitiesBFCL/parallel_41: two find_music_instrument_store calls — San Francisco and Chicago; instruments {Yamaha acoustic guitar, Kawai piano} in any order and either casing.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 511 OUT889
✓	FindRockAndJazzConcertsInTwoCitiesBFCL/parallel_43: four concert_finder calls — {San Francisco, New York} x {rock, jazz} over the next month. time_period may be omitted (defaults to 30 days ≈ "next month"). Isolated from concert_find_nearby (parallel_44) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 806 OUT1 481
✓	GetExhibitionDetailsForTwoMuseumsBFCL/parallel_40: two museum_exhibition_detail calls — 'Wonder of Nature' at the Louvre and 'Age of Reptiles' at the British Museum. visitor_type {child, adult} (unordered). Tightened per methodology: the prompt explicitly asks for cost "for children and adult", so both visitor types are required (schema default is ['adult'] only).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 548 OUT3 165
✓	GetMostPlayedPopAndRockSongsBFCL/parallel_45: two musicCharts_getMostPlayed calls — most played Pop and Rock song in Australia. Optional duration omitted (ground truth accepts the schema default of 0).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 470 OUT2 066
✓	GetMuseumInfoForTwoCitiesBFCL/parallel_39: two museum_info_get_info calls — Washington D.C. and Paris; details {Opening hours, Adult tickets, Child tickets} in any order.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 500 OUT1 978
✓	GetRgbAndHexForThreeColorsBFCL/parallel_37: three color_converter_get_color_info calls — navy, purple, maroon; each with {RGB, HEX} (unordered).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 622 OUT1 965
×	GetTeamRankingsInTwoLeaguesBFCL/parallel_47: two get_team_ranking calls — Barcelona in UEFA Champions League and Manchester United in La Liga.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: match0 == 1 Values: match0 = 0	Failed	IN7 471 OUT1 046
×	MixPaintColorsForTwoRequestsBFCL/parallel_36: two paint_color_mixture calls — watercolor magenta and acrylic navy.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN14 200 OUT1 689
✓	WinningPercentageForTwoTeamsTwoSeasonsBFCL/parallel_46: four calculate_winning_percentage calls — {Lakers, Bulls} x {2018, 2020}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 766 OUT1 603
Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 1) 11 passed2 failed	84.6%	IN121 977 OUT32 789
✓	AverageFourNumberSetsBFCL/parallel_132: four calculate_average calls — one per list of numbers (element order preserved).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 764 OUT1 173
×	CalculateBmiFourPeopleMetricBFCL/parallel_134: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt states "All measurements are in the metric system", so unit == "metric" is required (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: match0 == 1 Values: match0 = 0	Failed	IN8 754 OUT1 570
×	CalculateCityDistancesInKilometersBFCL/parallel_135: three geo_distance_calculate calls — NY->LA->Miami->NY, in kilometers. Tightened: the tool defaults to miles but the prompt asks "in kilometers", so units == "kilometers" is REQUIRED (omission would return miles, contradicting the request — the ground truth's "omit" option is unsafe here).1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN18 795 OUT7 927
✓	CalculateGpsDistancesAlongRouteBFCL/parallel_133: three calculate_distance calls over GPS coordinate pairs (Paris->Rome->Athens-> Giza), each in kilometers. Uses the coordinate-based tool (coord1/coord2), not the city-name distance tools that are also visible.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 790 OUT3 697
✓	CompareAverageTemperatureNyAndLaBFCL/parallel_128: two average_temperature calls — New York 7 days in Fahrenheit, Los Angeles 7 days in Celsius. Tightened: the prompt explicitly requests Fahrenheit for NY, so temp_unit == "Fahrenheit" is required (also the default); Celsius is non-default and required for LA.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 358 OUT4 155
✓	CreateTwoHistogramsWithFiveBinsBFCL/parallel_129: two create_histogram calls — one per data set, 5 bins each (data order preserved).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 822 OUT1 223
✓	EstimateElephantPopulationThreeScenariosBFCL/parallel_125: three elephant_population_estimate calls — current 500 at 2%, 1.5%, 2.5% over 10y.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 699 OUT1 313
✓	FindItalianRestaurantsInNyAndLaBFCL/parallel_130: two find_restaurants calls — New York and Los Angeles, Italian, 4 results each, dietary_requirements [vegan, gluten-free]. Distinct from restaurant_find_nearby (this tool carries food_type + number, which the prompt supplies, so it is the correct choice here).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 422 OUT1 182
✓	FindRestaurantsByDietInThreeCitiesBFCL/parallel_127: three restaurant_find_nearby calls — New York, Los Angeles, Chicago, each with dietary_preference [Vegan, Gluten-free, Dairy-free]. List order follows the prompt (matches ground truth).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 545 OUT2 199
✓	PredictEvolutionDarwinAndLamarckBFCL/parallel_126: two prediction_evolution calls — African Elephant over 5000y, Darwin then Lamarck. Tightened per methodology: the prompt explicitly says "using the Darwin model", so model == "Darwin" is required for the first call (ground truth also accepts omission via the 'Darwin' default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 381 OUT1 777
✓	ProjectPopulationGrowthJapanAndIndiaBFCL/parallel_124: four population_projections calls — Japan/10y and India/20y, each at the current (default) growth rate and at an explicit rate. "current growth rate" maps to the default (growth_rate omitted, accepts null or 0.01); the explicit 1.5%/2.1% rates are required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 706 OUT2 755
✓	RouteThreeLegsAvoidingTollsBFCL/parallel_131: three map_routing_fastest_route calls — SF->Palo Alto->San Jose->home, all avoiding tolls. Tightened: the prompt explicitly says "avoiding toll roads", so avoid_tolls == true is required for every leg (non-default value stated in the prompt).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 551 OUT1 428
✓	ShortestCityDistanceBusWithAndWithoutTransferBFCL/parallel_136: two city_distance_find_shortest calls — NY->LA by bus, first without transfer then allowing transfer. Tightened: the prompt explicitly says "using a bus", so transportation == "bus" is required for both (also the default). allow_transfer: false/omitted for the first, true for the second.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 390 OUT2 390
Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 2) 13 passed1 failed	92.9%	IN126 112 OUT20 863
✓	CalculateAreaOfThreeTrianglesBFCL/parallel_143: three calc_area_triangle calls — (10,5), (15,7), (20,10).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 958 OUT1 569
✓	CalculateBinomialProbabilityBasketballBFCL/parallel_161: three calculate_binomial_probability calls — basketball shooter at 60%: (10,7), (15,10), (20,15). probability_of_success == 0.6 is required (prompt states 60%). ISOLATED here (away from prob_dist.binomial in ToolCalling-Parallel-D3): both tools compute binomial probability, and this prompt does not name a function, so having both visible would make the choice genuinely ambiguous. Kept apart so exactly one correct tool is available.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 037 OUT2 674
✓	CalculateBmiInMetersForThreePeopleBFCL/parallel_138: three calculate_BMI calls (weight kg, height meters) — John, Sarah, Mike.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 019 OUT1 204
✓	CalculateCelestialDistancesInMilesBFCL/parallel_149: two calculate_distance calls between celestial bodies (Mars-Venus, Mars-Jupiter), in miles. Uses the body-name distance tool (body1/body2). Tightened: prompt asks "in miles", so unit == "miles" is required (tool default is kilometers, so omission would be wrong).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 764 OUT1 245
✓	CalculateClockAnglesAtThreeTimesBFCL/parallel_145: three calculate_clock_angle calls — 3:15, 8:20, 11:50. Tightened: the prompt asks for results "rounded to 2 decimal places", so round_to == 2 is required (also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 939 OUT2 499
✓	CalculateFactorialOfFourNumbersBFCL/parallel_144: four math_factorial calls — 5, 3, 4, 2.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 968 OUT890
✓	CalculateLightTravelTimeThreeLegsBFCL/parallel_147: three light_travel_time calls — 4.22, 6.1, 5.88 light years. Tightened: the prompt explicitly states the speed of light (299792458 m/s), so speed_of_light == 299792458 is required (also default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 025 OUT1 708
✓	CalculateSpeedForCarBicycleTrainBFCL/parallel_148: three calculate_speed calls — car 500m/25s in km/h, bicycle 1000m/200s in m/s, train 10000m/600s in km/h. Each output unit is explicitly named in the prompt, so to_unit is required.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 040 OUT1 197
✓	FetchAllDataFieldsForTwoEmployeesBFCL/parallel_139: two employee_fetch_data calls — Tech Solutions, employees 12345 and 67890, each fetching all four data fields (order follows the prompt / ground truth).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 872 OUT517
×	FindDiCaprioDramaAndComedyMoviesBFCL/parallel_140: two imdb_find_movies_by_actor calls — Leonardo DiCaprio, Drama in 2010 and Comedy in 2012. Tightened: the prompt explicitly names each category, so category is required for both.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN9 082 OUT2 894
✓	GetImaxAndTwoDTheaterReleasesBFCL/parallel_141: two get_theater_movie_releases calls — New York IMAX over 7 days, Los Angeles 2D over 14 days. Tightened: the prompt explicitly names each format, so format is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 789 OUT1 097
✓	PlotTwoSineWavesBFCL/parallel_146: two plot_sine_wave calls with all five parameters supplied by the prompt.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 895 OUT577
✓	SortFourListsAscendingAndDescendingBFCL/parallel_137: eight array_sort calls — four lists, each sorted ascending AND descending (list element order preserved as given).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 844 OUT2 310
✓	UpdateTwoCustomersNameAndEmailBFCL/parallel_142: two update_user_info calls — users 12345 and 67890, both set name "John" and email "example@.com". update_info is a nested object. Tightened: the prompt names the 'CustomerInfo' database, so database == "CustomerInfo" is required (also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 880 OUT482
Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 3) 10 passed1 failed	90.9%	IN87 936 OUT16 255
✓	AreaUnderCurveForTwoPolynomialsBFCL/parallel_150: two mathematics_calculate_area_under_curve calls — polynomial [3,-2,1] over [-1,2] and [1,0,-1] over [0,3] (coefficient/limit order preserved).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 831 OUT2 039
✓	BinomialProbabilitiesThreeGamesBFCL/parallel_160: three prob_dist_binomial calls — (20,5), (50,15), (100,30), each with p == 0.3 (prompt states a 0.3 win probability). The prompt explicitly names 'prob_dist.binomial'; the colliding calculate_binomial_probability tool is kept in a separate script (ToolCalling-Parallel-D2).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 001 OUT2 803
✓	CalculateBmiSystemFourPeopleBFCL/parallel_154: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt says "use the metric system for all calculations", so system == "metric" is required (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 128 OUT911
✓	DiceRollProbabilitiesThreeScenariosBFCL/parallel_159: three probability_dice_roll calls — roll 4 three times and roll 2 twice on a six-sided die (die_sides default 6, accepts omission), then roll 7 twice on an eight-sided die (die_sides == 8 required, non-default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 958 OUT2 007
✓	GenerateNormalVariatesTwoConfigsBFCL/parallel_158: four random_normalvariate calls — two at (mu 5, sigma 2) and two at (mu 10, sigma 3). Duplicate calls: each configuration must appear exactly twice (multiplicity 2).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 064 OUT1 161
✓	PredictPricesForFourHousesBFCL/parallel_157: four predict_house_price calls — New York (3/2/1500), Los Angeles (4/3/2000), Chicago (2/1/1200), Miami (3/2/1800).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 252 OUT2 608
✓	RaiseTwoNumbersToPowersBFCL/parallel_152: two math_power calls — 3^5 and 2^3, no modulus (mod omitted / null).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 684 OUT732
✓	RunLinearRegressionWithAndWithoutStandardizationBFCL/parallel_155: two run_linear_regression calls — same predictors [Age, Income, Education] and target "Spending Score", first without standardization then with. standardize is explicitly requested (false then true), so it is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 876 OUT805
✓	TrainRandomForestFourConfigsBFCL/parallel_156: four random_forest_train calls on "data_random_forest" — (100,10), (200,20), (300,30), (400,40). The prompt explicitly names this function, disambiguating it from train_random_forest_classifier visible in this script.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 310 OUT1 307
✓	TrainRfClassifierTwoConfigsBFCL/parallel_153: two train_random_forest_classifier calls — dataset1 (depth 10, 100 trees) and dataset2 (depth 20, 200 trees). The prompt explicitly names this function, disambiguating it from the other random-forest training tool (random_forest_train) visible in this script.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 874 OUT556
×	TriangleAreasInDifferentUnitsBFCL/parallel_151: three geometry_area_triangle calls in different units — (15,20) square meters, (25,30) square feet, (35,40) square inches. First unit is the default (accepts omission / square meters); feet and inches are non-default and required.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: match0 == 1 Values: match0 = 0	Failed	IN7 958 OUT1 326
Tool Calling - Parallel Calls: Finance, Stats & Legal 13 passed	100.0%	IN114 989 OUT24 332
✓	BuildTwoDecisionTreeClassifiersBFCL/parallel_12: two model_DecisionTreeClassifier calls — gini/depth 5 and entropy/depth 10, both random_state 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 440 OUT634
✓	CalculateFutureValueForThreeTermsBFCL/parallel_18: three financials_calculate_future_value calls — years 1, 5, 10 (same 1000 @ 0.05).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 602 OUT1 945
✓	CalculatePresentValueForThreeTermsBFCL/parallel_14: three calculate_present_value calls — years 10, 20, 30 (same payment 1000, rate 0.05).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 599 OUT1 988
✓	CapitalGainsTaxForTwoStatesBFCL/parallel_15: two calculate_capital_gains_tax calls — CA (15000/25000) and FL (20000/50000).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 492 OUT2 441
✓	ComputePythagorasForTwoTrianglesBFCL/parallel_10: two math_pythagoras calls — sides (3,4) and (5,12).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 383 OUT1 383
✓	ConfidenceIntervalForTwoSampleSizesBFCL/parallel_13: two confidence_interval_calculate calls — sample sizes 50 and 150. Tightened per methodology: the prompt explicitly requests "95% confidence interval", so confidence_level == 0.95 is required (schema default is 0.9, but the prompt states the value).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 495 OUT2 989
✓	CourtCaseStatusAndTrialDatesBFCL/parallel_22: four court_info_get_case_status calls — cases XY1234 (LA) and GH5678 (Orange County), each queried for status (details may be omitted via the 'status' default) and for trial_date.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN13 028 OUT3 345
✓	GetStockPriceAndVolumeForThreeCompaniesBFCL/parallel_17: three get_stock_data calls — AAPL, GOOG/GOOGL, MSFT; each with data_points {price, volume} (unordered).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 534 OUT1 133
✓	LoanEligibilityForTwoInstitutionsBFCL/parallel_20: two loan_eligibility_check calls — HSBC (500k/100k) and Wells Fargo (700k/120k).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 507 OUT1 067
✓	MortgagePaymentForThreeLoanTermsBFCL/parallel_19: three calculate_mortgage_payment calls — loan terms 15, 20, 30 (400000 @ 0.04).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 627 OUT3 760
✓	PredictHousePriceInTwoCitiesBFCL/parallel_11: two ml_predict_house_price calls — New York 3000 sq ft, Los Angeles 4000 sq ft.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 414 OUT1 425
✓	ReturnOnInvestmentGainAndLossBFCL/parallel_16: two calculate_return_on_investment calls — gain 500 on 2000, loss -1000 on 5000.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 445 OUT933
✓	SearchMoneyLaunderingConvictionsBFCL/parallel_21: two law_crimes_search calls — money laundering in San Francisco 2019 and Texas 2018.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 423 OUT1 289
Tool Calling - Parallel Calls: Legal, History & Environment 11 passed2 failed1 skipped	84.6%	IN127 322 OUT27 555
✓	BookTwoConnectingFlightsBFCL/parallel_30: two book_flight calls — San Francisco→Tokyo (2022-05-03) and Tokyo→Sydney (2022-05-18).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 057 OUT615
✓	CalculateAlimonyForTwoDurationsBFCL/parallel_23: two alimony_calculator_ca_calculate calls — durations 10 and 20 years (10000/3000 income).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 120 OUT4 485
×	CalculateWasteForFamilyAndBachelorBFCL/parallel_29: two waste_calculation_calculate calls — LA family (2 adults, 2 children) and New York bachelor (1 single). The population argument is a nested object.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: match1 == 1 Values: match1 = 0	Failed	IN9 070 OUT3 176
✓	FetchTwoHistoricalFactsBFCL/parallel_31: two history_fact_fetch calls — Treaty of Paris and Magna Carta. Optional depth/year omitted (ground truth accepts the schema defaults).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 976 OUT1 092
✓	FindClassicalConcertsWithParkingBFCL/parallel_44: two concert_find_nearby calls — classical concerts near Berlin and Paris next Friday. Tightened per methodology: the prompt explicitly asks for concerts "with available parking", so amenities == ['Parking'] is required (this also matches the schema default, but the prompt states it). Isolated from concert_finder (parallel_43) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 034 OUT2 177
–	FindCourtCasesByNumberAndTypeBFCL/parallel_26: four court_case_find calls — the cross product of case numbers {67813, 71249} and types {Civil, Criminal}; case_number is a single-element array; Civil may be omitted via its default. FLAGGED as borderline (see report): the prompt "case numbers '67813','71249' for type 'Civil' and 'Criminal'" is ambiguous about the pairing; BFCL ground truth expects all four combinations, kept faithful.0 AI queries 0 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL Skipped: Ambiguous call decomposition (item flaw, not model behavior): 'case numbers 67813, 71249 for type Civil and Criminal' has no single deterministic reading - two calls (both numbers per type, case_number as a list) is as valid as the BFCL 2x2 = four single-number calls; a competent human would not answer deterministically. Kept for traceability.	Skipped	IN0 OUT0
✓	FindNatureReservesNearTwoCitiesBFCL/parallel_27: two nature_reserve_find_nearby calls — Berkeley (10km, picnic tables + restrooms) and Tokyo (5km, playgrounds + biking trails). Amenity list order is unordered.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 058 OUT736
✓	GetCaseDetailsWithHistoryAndLitigantsBFCL/parallel_24: two law_case_get_details calls — cases 28473 and 64725. Tightened per methodology: the prompt explicitly asks for "their history and details of litigants", so include_history == true and include_litigants == true are required (schema defaults are false).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 040 OUT1 421
✓	GetEventsForTwoPresidenciesBFCL/parallel_32: two us_history_events_by_presidency calls — Abraham Lincoln and George Washington. Optional start_year/end_year omitted (ground truth accepts the schema defaults).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 971 OUT1 329
✓	GetPresidentsAndVicePresidentsByYearBFCL/parallel_33: four get_president_and_vp calls — president for 1980 and 2016, vice president for 1975 and 2011.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 326 OUT2 116
✓	GetReligionInfoForTwoEmpiresBFCL/parallel_35: two ancient_empires_get_religion_info calls — Persian Empire and Mauryan Empire. Tightened per methodology: the prompt explicitly asks for "religious history and influences", so include_influences == true is required (schema default is false).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 011 OUT2 428
✓	GetTemperatureForTwoCitiesBFCL/parallel_28: two get_current_and_future_temperature calls — Seattle and Los Angeles, next 3 hours.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 983 OUT1 010
×	LookupLawsuitsForTwoYearsBFCL/parallel_25: two lawsuit_lookup calls — Dara Inc for years 2019 and 2018.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN18 622 OUT3 210
✓	TrackChristianityInTwoRegionsBFCL/parallel_34: two religion_history_track calls — Christianity in Egypt and Turkey, 100–1500 A.D.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 054 OUT3 760
Tool Calling - Parallel Calls: Markets & Environment 12 passed1 failed	92.3%	IN114 500 OUT30 654
✓	CalculateCagrForTwoInvestmentsBFCL/parallel_176: two calculate_cagr calls — (5000,7000,5) and (8000,12000,3).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 665 OUT1 888
✓	CalculateEmissionSavingsForSolarAndWindBFCL/parallel_186: two calculate_emission_savings calls — solar(12 months) and wind(8 months), Midwest region. Prompt states the Midwest region explicitly -> region required (accept "Midwest"/"Midwest region").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 558 OUT3 153
✓	CompareFutureValueWithAndWithoutContributionsBFCL/parallel_182: two finance_calculate_future_value calls — (5000,0.07,10, no contribution) and (3000,0.06,10, contribution 200). First scenario states "no additional contributions" -> null or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 725 OUT2 630
×	ComputeEuclideanNormForThreePointsBFCL/parallel_183: three math_hypot calls — (5,7), (10,15), (20,25). z omitted or 0 (2D points).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 2 expectedCount = 3	Failed	IN8 620 OUT2 714
✓	EstimatePopulationsForTwoSpeciesAcrossTwoYearsBFCL/parallel_185: four estimate_population calls — Bengal Tigers/India and African Elephants/Kenya for 2021 and for 2023. NOTE: upstream ground truth's third entry lists year "" (omitted -> default 2020), which is an upstream error: the prompt explicitly asks for 2023 for "both these species". We tighten both 2023 estimates to year == 2023 (fair deterministic answer).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 904 OUT2 169
✓	FindRootsForTwoQuadraticEquationsBFCL/parallel_184: two algebra_quadratic_roots calls — (3,7,2) and (5,-4,1).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 678 OUT1 430
✓	GetClosingPricesForTwoCompaniesOnTwoDatesBFCL/parallel_178: four get_stock_price (by-date variant) calls — the 2x2 cross product of {Microsoft, Apple} x {2022-01-01, 2022-02-01} on NASDAQ. Upstream ground truth's first entry loosely lists company_name as [Microsoft, Apple]; the deterministic human answer is the four distinct (company, date) combinations, which we require. exchange NASDAQ (default) may be omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 862 OUT4 161
✓	GetDetailedAirQualityForThreeDaysBFCL/parallel_187: three get_air_quality calls — New York City, detail true, for 2023-05-05, 05-04, 05-03. Prompt asks for PM2.5/PM10/ozone/sources -> detail == true; "today/yesterday/day before" with today = May 5, 2023.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 764 OUT1 431
✓	GetFourPriceTypesForTwoCompaniesBFCL/parallel_180: eight stock_price calls — {Microsoft, Apple} x {Open, Close, High, Low}, days 30. Prompt explicitly lists all four data types, so data_type must be set explicitly (required) to distinguish the eight calls; omission (default 'Close') cannot represent the four requested types.1 AI query 8 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL	Passed	IN9 438 OUT1 798
✓	GetPricePerOunceForFourMetalsBFCL/parallel_177: four get_metal_price calls — gold, silver, platinum, palladium, measure 'ounce'. Prompt asks "price per ounce" -> measure == "ounce" required.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 785 OUT2 085
✓	GetStockPricesForFourCompaniesAndDurationsBFCL/parallel_181: four get_stock_prices calls — Apple(1 week), Microsoft(2 weeks), Amazon(3 weeks), Tesla(1 month). Each 'companies' argument is a single-element list.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 834 OUT1 347
✓	GetTrafficInfoForThreeLegsWithModesBFCL/parallel_188: three get_traffic_info calls — NY->LA (driving), LA->SF (bicycling), SF->NY (transit). First leg explicitly says "driving" -> mode == "driving" required (tightened; also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 759 OUT1 983
✓	ProjectFutureValueForThreeStocksBFCL/parallel_175: three portfolio_future_value calls — AAPL(5000,0.07,5), MSFT(8000,0.06,7), AMZN(10000,0.08,10).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 908 OUT3 865
Tool Calling - Parallel Calls: Mathematics & Physics (set 1) 13 passed	100.0%	IN104 045 OUT20 125
✓	AreaOfTwoRightTrianglesInMetersBFCL/parallel_68: two calculate_area calls — right triangles base/height 12x15 and 18x24 in meters. unit required (tool default is 'cm'; the prompt states meters/square meters).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 988 OUT1 025
✓	CheckPlayerStatusAcrossThreeSeasonsBFCL/parallel_49: three player_status_check calls — player 3142 in RocketLeague for seasons 2017/2018/2019. Seasons are explicitly given, so each is required (default 2023 not applicable).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 071 OUT940
✓	CircumferenceOfFourCirclesInCmBFCL/parallel_65: four calculate_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt gives every radius in cm (e.g. "5cm") and the tool default unit is 'm', so a correct call must pass unit == 'cm'/'centimeter' (ground truth requires it on the first call and merely allows omission on the rest; we require it consistently since it is stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 213 OUT2 043
✓	ConvertEuroAndPoundsToDollarsBFCL/parallel_59: two get_conversion_cost calls — 15000 Euro->dollars and 200 pounds->dollars.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 900 OUT2 559
✓	FinalVelocityForFourDropsBFCL/parallel_81: four calculate_final_velocity calls — heights 10/20/15/25, initial_velocity 0 (stated), gravity 9.81 (default, may be omitted).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 398 OUT2 718
✓	FindItalianAndJapaneseRestaurantsBFCL/parallel_52: two restaurant_finder calls — Italian in New York (preferences ['Vegetarian']) and Japanese in Los Angeles (delivery; ['Delivery'] or omitted since default is ['Delivery']).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 896 OUT1 536
✓	FindRootsOfTwoQuadraticsAlgebraBFCL/parallel_62: two algebra_quadratic_roots calls — 3x^2+4x+2 and 5x^2-7x+3.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 050 OUT2 279
✓	FindSoupAndVeganSaladRecipesBFCL/parallel_51: two recipe_search_find calls — Chicken Noodle Soup (diet omitted/default 'Keto') and a Vegan Salad (diet == 'Vegan', dish name accepts several phrasings).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 831 OUT860
✓	GcdOfJohnAndMaryNumbersBFCL/parallel_80: two math_gcd calls — gcd(36,48) and gcd(60,96). Same tool as parallel_76.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 946 OUT1 465
✓	GcdOfTwoPairsBFCL/parallel_76: two math_gcd calls — gcd(45,60) and gcd(81,27).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 878 OUT1 063
✓	GetPokemonMovesAndCheckSolarBeamBFCL/parallel_48: two PokemonGO_get_moves calls — Pikachu's move list (move omitted/default 'Run') and a check whether Bulbasaur can learn 'Solar Beam'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 868 OUT1 236
✓	PrimeFactorsFormattedAndArrayBFCL/parallel_74: four get_prime_factors calls — numbers 4567 and 7890, each formatted true and false (user wants "a formatted string as well as an array").1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 160 OUT2 026
✓	SaveGameProgressEasyAndHardBFCL/parallel_50: two game_save_progress calls — stage 7 easy and stage 3 hard; level omitted/default 'user'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 846 OUT375
Tool Calling - Parallel Calls: Mathematics & Physics (set 2) 13 passed	100.0%	IN107 921 OUT24 667
✓	AreaOfThreeCirclesInMetersBFCL/parallel_66: three geometry_area_circle calls — radii 5/10/15 in meters (default is meters, so units may be omitted).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 335 OUT1 257
✓	BookTwoHotelRoomsBFCL/parallel_57: two hotel_booking_book calls — Sheraton (NY, 2 adults/1 child) and Marriott (LA, 1 adult/2 children).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 383 OUT880
✓	CheckAvailabilityForTwoHotelsBFCL/parallel_56: two hotel_booking_check_availability calls — Queens Hotel (Berlin) and Royal Hotel (Paris), each with check-in/out dates (several date formats accepted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 315 OUT1 119
✓	CircumferenceOfFourCirclesInMetersBFCL/parallel_70: four geometry_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt asks for output in meters and gives radii in m, but the tool default unit is 'cm', so a correct call must pass units == 'm'/'meters' (ground truth requires it on the first call and only allows omission on the rest; we require it consistently since it is explicitly stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 515 OUT2 387
✓	DerivativeOfPolynomialAndItsDerivativeBFCL/parallel_71: two calculate_derivative calls — 3x^3-2x^2+5x-7 at x=4, then its derivative 9x^2-4x+5 at x=2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 230 OUT3 714
✓	FinalVelocityForThreeDragRaceRoundsBFCL/parallel_83: three final_velocity calls — (v0=0,a=5,t=10), (v0=10,a=7,t=8), (v0=20,a=4,t=12).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 512 OUT1 346
✓	FindGroceryStoresByCriteriaBFCL/parallel_55: two grocery_store_find_by_criteria calls — NYC ['24 hours'] and San Diego ['Home Delivery'].1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 148 OUT1 569
✓	GetTwoCookingRecipesBFCL/parallel_53: two get_cooking_recipe calls — Lasagne Bolognese for 4 and Caesar Salad for 2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 166 OUT2 728
✓	GetTwoExchangeRatesBFCL/parallel_58: two get_exchange_rate calls — USD->AUD and USD->CAD.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 130 OUT1 039
✓	HighestCommonFactorOfFourPairsBFCL/parallel_77: four math_hcf calls — (45,60), (90,120), (36,48), (72,96). FLAGGED (see final report): the prompt is phrased as chaining ("use that result to find the HCF with...") but the ground truth treats it as four INDEPENDENT parallel calls on the literal pairs; a human might interpret it as sequential composition. Kept BFCL-faithful (four independent literal-pair calls) since parallel semantics preclude feeding one call's result into another.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 549 OUT2 843
✓	OrderSameItemsFromTwoWholeFoodsLocationsBFCL/parallel_54: two whole_foods_order calls — same two items (order-independent) at downtown and uptown, size large.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 196 OUT725
✓	PrimeFactorizeAsDictionaryBFCL/parallel_79: two prime_factorize calls — 4567 and 7890, return_type 'dictionary' (explicitly requested).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 172 OUT3 628
✓	SolveTwoQuadraticEquationsBFCL/parallel_63: two solve_quadratic_equation calls — (5,6,1) and (3,2,1). Prompt names this function explicitly.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 270 OUT1 432
Tool Calling - Parallel Calls: Mathematics & Physics (set 3) 12 passed	100.0%	IN98 280 OUT17 972
✓	AreaOfTwoCirclesInMetersBFCL/parallel_67: two geometry_calculate_area_circle calls — radii 5 and 10 in meters. unit required (tool default is 'cm'; the prompt states meters).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 019 OUT1 134
✓	AreaOfTwoTrianglesBFCL/parallel_69: two calculate_triangle_area calls — base/height 10x5 and 8x6.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 079 OUT1 167
✓	DerivativeOfTwoFunctionsAtValuesBFCL/parallel_73: two calculus_derivative calls — 3x^2+2x-1 at 5 (variable x, default) and 4y^3-3y^2+2y-1 at 3 (variable y).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 124 OUT1 171
✓	DisplacementForThreeCarsBFCL/parallel_84: three calculate_displacement calls — (15,7,accel 3.5), (20,10,accel 2.0), (25,8,no acceleration). The third car travels "without any acceleration" so acceleration == 0 (also the default; omission accepted).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 349 OUT3 003
✓	EuclideanNormForThreePointsBFCL/parallel_61: three math_hypot calls — (3,4) and (6,8) with z omitted/0, and (9,12,15).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 254 OUT1 267
✓	FactorialOfThreeNumbersBFCL/parallel_60: three math_factorial calls — 5, 7, 9.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 081 OUT1 006
✓	FinalSpeedForTwoFreeFallObjectsBFCL/parallel_85: two calculate_final_speed calls — initial_speed 0 for 10 s and initial_speed 5 for 7 s; gravity -9.81 (default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 147 OUT2 262
✓	GcdViaNumberTheoryBFCL/parallel_78: two number_theory_gcd calls — (45,60) and (81,63). Prompt names this function explicitly.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 022 OUT745
✓	IntegrateTwoFunctionsWithTwoMethodsBFCL/parallel_72: four integrate calls — x3 on [2,5] and 2x2+3x-1 on [-1,3], each with trapezoid (default, may be omitted) and simpson (explicit).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 468 OUT2 063
✓	PrimeFactorsOfThreeNumbersBFCL/parallel_75: three number_analysis_prime_factors calls — 45, 100, 150.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 087 OUT993
✓	SolveFourQuadraticsWithRootTypeBFCL/parallel_64: four solve_quadratic calls — (2,5,3) all roots, (1,-3,2) real, (4,-7,3) all roots, (1,2,1) real. 'all' is the default so it may be omitted; 'real' is explicitly requested and required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 538 OUT2 183
✓	VelocityForTwoCyclingDaysBFCL/parallel_82: two calculate_velocity calls — 120 km in 5 h and 150 km in 6 h; unit km/h (default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 112 OUT978
Tool Calling - Parallel Calls: Mixed Domains 9 passed1 failed	90.0%	IN73 363 OUT16 314
✓	CalculateBmiForTwoPeopleBFCL/parallel_4: two calculate_bmi calls — 6 ft / 80 kg and 5.6 ft / 60 kg.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 278 OUT2 057
✓	CalculateEmForceForTwoTimeIntervalsBFCL/parallel_1: two calculate_em_force calls — same field/area, d_time 4 then 10.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 299 OUT1 237
✓	CalculateFactorialForThreeNumbersBFCL/parallel_7: three math_factorial calls — 5, 10, 15.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 338 OUT1 268
✓	CalculateResistanceForCopperAndAluminumBFCL/parallel_2: two calculate_resistance calls — copper then aluminum, same length/area. Tightened per methodology: the prompt explicitly names "resistivity of copper", so the first call must pass resistivity == "copper" (ground truth also accepts omission via the 'copper' default, but an explicitly-stated value must be supplied). Aluminum is non-default → required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 286 OUT1 455
✓	CalculateSalesTaxForThreeCitiesBFCL/parallel_6: three calculate_sales_tax calls — Chicago/IL $30.45, Sacramento/CA $52.33, Portland/OR $11.23.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 470 OUT1 420
✓	FetchPopulationForCitiesStateAndCountryBFCL/parallel_8: four database_us_census_get_population calls — NYC (city), LA (city), Alaska (state), USA (country); year defaults to 2000 (may be omitted). Requires area-type inference.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 554 OUT1 725
×	FindMovieShowingsForTwoMoviesBFCL/parallel_9: two find_movie_showing calls near San Diego — Tenet at 5 pm and No Time To Die at 7:30 pm.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 1 expectedCount = 2	Failed	IN7 158 OUT2 101
✓	GetSequenceAnd3DForThreeProteinsBFCL/parallel_3: three protein_info_get_sequence_and_3D calls — HbA1c, normal + rat hemoglobin. Tightened per methodology: the prompt explicitly asks for "their 3D models", so model_3d == true is required (ground truth also accepts omission via the default of true).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 417 OUT2 628
✓	ListShowRatingsOnNetflixAndHuluBFCL/parallel_5: two streaming_services calls — Netflix['Friends'] and Hulu['The Office','Stranger Things'] (Hulu show order unordered), both sorted by rating. Must group shows per service.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 301 OUT2 034
✓	PlayTwoArtistsForGivenDurationsBFCL/parallel_0: two spotify_play calls — Taylor Swift for 20 min and Maroon 5 for 15 min.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 262 OUT389
Tool Calling - Parallel Calls: Physical & Life Sciences (set 1) 12 passed	100.0%	IN102 016 OUT26 387
✓	ElectricFieldForTwoChargesBFCL/parallel_94: two calculate_electric_field calls — charge 5/distance 2 and charge 3/distance 4. "Conducted in a vacuum" = the default permitivity → may be omitted or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 410 OUT1 786
✓	ElectromagneticForceInTwoMediaBFCL/parallel_96: two electromagnetic_force calls — same charges 5,10 and distance 2, differing by medium_permittivity. Vacuum call (8.854e-12) equals the default → may be omitted or 8.854e-12; the second medium (5e-12) is non-default → required.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 566 OUT2 712
✓	EstimateTravelTimeForThreeRoutesBFCL/parallel_93: three route_estimate_time calls — (1) NYC->Miami via Philadelphia/Washington D.C./Atlanta, (2) NYC->Miami skipping Atlanta (via Philadelphia/Washington D.C.), (3) Philadelphia-> Miami via Washington D.C. Stop orderings from ground truth accepted. FLAG parallel_93: BORDERLINE. Ground truth allows many stop-list variants, some including the start city itself as a stop; a competent human's stop set is not fully deterministic. Kept BFCL-faithful by accepting the natural stop orderings (start city excluded from stops).1 AI query 3 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL	Passed	IN8 561 OUT3 795
✓	FinalSpeedForTwoDroppedObjectsBFCL/parallel_88: two calculate_final_speed calls — v0 0/height 10 and v0 5/height 20. gravity stated as 9.8 (the default) so it may be omitted or 9.8 per ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 416 OUT2 283
✓	FinalVelocityForTwoCarsBFCL/parallel_87: two calculate_final_velocity calls — Car A (v0 0, a 6, t 10), Car B (v0 20, a 4, t 15).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 433 OUT1 305
✓	FinalVelocityFromDistanceForTwoObjectsBFCL/parallel_86: two kinematics_final_velocity_from_distance calls — accel 5/dist 100 and accel 10/dist 200. "Both objects start from rest" → initial_velocity may be omitted or 0 (default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 405 OUT1 941
✓	FindVeganRestaurantsInThreeCitiesBFCL/parallel_91: three vegan_restaurant_find_nearby calls — LA, SF, Seattle, each with operating_hours 22 (prompt: "open until at least 22:00", a non-default value → required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 462 OUT1 505
✓	GenerateFourTravelItinerariesBFCL/parallel_90: four travel_itinerary_generator calls — Tokyo/7/200/urban, Paris/10/150/history, Sydney/5/100/nature, Rome/12/180/culture. Tokyo's "urban" equals the default → omit or "urban".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 787 OUT2 862
✓	GetDirectionsForRoundTripBFCL/parallel_89: three get_directions calls — SF->Palo Alto (fastest), Palo Alto->Golden Gate Bridge (scenic), Golden Gate Bridge->SF (fastest). route_type explicitly stated each time.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 512 OUT1 836
✓	MagneticFieldForTwoLoopsBFCL/parallel_95: two calculate_magnetic_field calls — current 10/radius 0.5 and current 15/radius 1.0. "Same as in free space" = default permeability → may be omitted or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 514 OUT1 618
✓	ResonantFrequencyForTwoCircuitsBFCL/parallel_97: two calculate_resonant_frequency calls — L 0.005/C 1e-07/round 3 and L 0.007/C 2e-07/round 4. Rounding is stated explicitly → required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 480 OUT2 725
✓	ShortestDrivingDistanceRoundTripBFCL/parallel_92: three get_shortest_driving_distance calls — NYC->LA, LA->Miami, Miami->NYC, all in miles (prompt explicitly "in miles" → non-default unit required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 470 OUT2 019
Tool Calling - Parallel Calls: Physical & Life Sciences (set 2) 12 passed1 failed	92.3%	IN112 061 OUT28 026
✓	AbsolutePressureAtTwoAltitudesBFCL/parallel_102: two calc_absolute_pressure calls — gauge 2.5 with atm 1 (default; may be omitted) and gauge 2.5 with atm 0.85 (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 519 OUT3 570
×	BoilingMeltingPointsAcrossSeaLevelsBFCL/parallel_100: four get_boiling_melting_points calls — water@0, iron@1000, water@1000, iron@0. FLAG parallel_100: BORDERLINE. The prompt ("water and iron at sea levels of 0 and 1000 meters respectively") reads most naturally as TWO calls (water@0, iron@1000), but the BFCL ground truth is the full 2x2 cross product (FOUR calls). Kept BFCL-faithful to the ground truth; a competent human could reasonably answer with two calls, so this item's expected call count is not fully deterministic.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN8 700 OUT1 923
✓	CellInfoDetailedAndBriefBFCL/parallel_108: two biology_get_cell_info calls — neuron with detailed=true (prompt: "detailed description"), muscle with detailed=false ("less detailed", the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 408 OUT2 801
✓	DensityForTwoSubstancesBFCL/parallel_101: two calculate_density calls (mass/volume schema) — 10 kg / 2 m³ and 15 kg / 3 m³. unit "kg/m³" equals the default → may be omitted or "kg/m³".1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 536 OUT1 158
✓	ElectricFieldStrengthAtSeveralDistancesBFCL/parallel_98: four calculate_electric_field_strength calls — charge 2, distances 0.5/1/2 in vacuum (default medium), plus distance 1 in air. The distance-1 vacuum and distance-1 air calls are distinguished by medium. Vacuum equals the default → may be omitted or "vacuum"; "air" is required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 837 OUT4 024
✓	EntropyChangeForTwoConditionsBFCL/parallel_103: two entropy_change_calculate calls — substance A, mass 2, temps 25->75 and 10->50. pressure 1 equals the default → may be omitted or 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 642 OUT1 744
✓	FetchDnaSequenceThreeWaysBFCL/parallel_106: three fetch_DNA_sequence calls for id XYZ123 — default fasta/0, then genbank format, then 500 bp upstream. Defaults (fasta, upstream 0) may be omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 613 OUT892
✓	FunctionLookupForTwoMoleculesBFCL/parallel_110: two cell_biology_function_lookup calls — ATP in mitochondria and DNA in nucleus, both with specific_function=true (prompt asks for a specific function within the organelle).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 469 OUT1 072
✓	HeatCapacityForThreeConditionsBFCL/parallel_105: three calc_heat_capacity calls — 300K/2.5m³, 350K/2.5m³, 300K/1.5m³, all for air. "air" equals the default gas → may be omitted or "air".1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 717 OUT1 573
✓	MolecularWeightForTwoCompoundsBFCL/parallel_111: two calculate_molecular_weight calls — Glucose (C6H12O6) and Sucrose (C12H22O11), both in grams/mole (prompt explicitly requests that unit; ground truth also accepts "g/mol").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 470 OUT1 339
✓	PhaseChangeEnergyForWaterBFCL/parallel_99: four thermo_calculate_energy calls — 500 g water, phase melting/freezing/ vaporization/condensation. Substance "water" equals the default → may be omitted or "water".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 879 OUT2 959
✓	ProteinSequencesForTwoGenesTwoSpeciesBFCL/parallel_107: four get_protein_sequence calls — BRCA1 and BRCA2 in Homo sapiens (default species, may be omitted) and in Pan troglodytes (non-default → required).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 708 OUT3 474
✓	ProteinsInThreeCompartmentsBFCL/parallel_109: three cellbio_get_proteins calls — nucleus/mitochondria/cytoplasm, each with include_description=true (prompt asks for "a brief description of each protein").1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 563 OUT1 497
Tool Calling - Parallel Calls: Physical & Life Sciences (set 3) 13 passed	100.0%	IN113 522 OUT24 543
✓	AnalyzeDnaForThreeMutationTypesBFCL/parallel_114: six analyze_dna_sequence calls — two sequences ("AGCTTAGCTA" and "AGCTTAGGCTA") against reference "AGCTTAGCTA", each checked for insertion (default, may be omitted), deletion, and substitution mutations.1 AI query 6 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL	Passed	IN9 201 OUT1 848
✓	DiabetesPredictionForFourPeopleBFCL/parallel_113: four diabetes_prediction calls for four people with distinct weight/height/activity.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 962 OUT1 673
✓	EntropyChangeIsothermalAndNotBFCL/parallel_104: two calculate_entropy_change calls — same 300->350 K and heat_capacity 4.18, once isothermal (the default true, may be omitted) and once not isothermal (false, required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 624 OUT3 593
✓	ForestGrowthForTwoRegionsBFCL/parallel_120: two forest_growth_forecast calls — Amazon Rainforest over 10 years and Boreal Forests of Canada over 20 years, excluding human impact (false = the default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 515 OUT2 429
✓	GenerateFourDnaSequencesBFCL/parallel_123: four generate_DNA_sequence calls — length 500 with a single-nucleotide preference of A, T, C, G respectively.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 822 OUT1 422
✓	GeneticSimilarityInTwoFormatsBFCL/parallel_115: four genetics_calculate_similarity calls — human vs chimpanzee and human vs gorilla, each in percentage (default, may be omitted) and fraction. species1 accepts "human"/"Human".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 794 OUT1 892
✓	GenotypeFrequenciesForThreeGenotypesBFCL/parallel_116: three calculate_genotype_frequency calls — allele frequency 0.7, genotypes AA/Aa/aa.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 679 OUT1 635
✓	IdentifyFourBirdsBFCL/parallel_119: four identify_bird calls — small blue/forest (default size, may be omitted), large black/lake, medium brown/desert, large green/tropical rainforest.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 918 OUT1 927
✓	MutationTypeForTwoSnpsBFCL/parallel_112: two mutation_type_find calls — rs123456 in Homo sapiens (default species, may be omitted) and rs7891011 in Canis lupus familiaris (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 537 OUT2 643
✓	PopulationDensityForChinaTwoYearsBFCL/parallel_117: two calculate_density calls (country/population schema) — China in 2000 (pop 1.267 billion) and 2010 (pop 1.341 billion), land area 9.597 million km² both times.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 608 OUT1 090
✓	PrecipitationStatsForThreeTimeFramesBFCL/parallel_118: three ecology_data_precipitation_stats calls — Amazon rainforest for six_months, year, five_years.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 610 OUT1 321
✓	TurtlePopulationForTwoYearsBFCL/parallel_121: two ecology_get_turtle_population calls — Galapagos Islands in 2015 and 2020, both with species info requested (species=true; years stated explicitly → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 530 OUT1 639
✓	VehicleEmissionsForThreeVehiclesBFCL/parallel_122: three calculate_vehicle_emission calls — 15000 miles each: gas with default factor 1.4 (may be omitted), diesel with factor 2.7, EV with factor 0.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 722 OUT1 431
Tool Calling - Parallel Calls: Services & Lookups 12 passed	100.0%	IN98 416 OUT20 469
✓	CalculateShortestAndScenicDistancesForTwoLegsBFCL/parallel_190: four calculate_shortest_distance calls — {NYC->LA, LA->Miami} x {Shortest, Scenic}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 368 OUT2 333
✓	FetchNewsForTwoTopicsInEuropeBFCL/parallel_192: two get_news calls — Climate Change and Artificial Intelligence, 5 articles each, region Europe. Region "Europe" explicitly requested -> required (accept "Europe"/"EU").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 049 OUT1 642
✓	FindLibrariesWithFacilitiesInThreeCitiesBFCL/parallel_191: three public_library_find_nearby calls — NY[Reading Room,Fiction], LA[Wi-Fi,Children Section], Chicago[Cafe,Reading Room]. Facilities compared as an unordered pair.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 267 OUT2 550
✓	FindParksWithAmenitiesInThreeCitiesBFCL/parallel_189: three parks_find_nearby calls — NY[Tennis Court,Picnic Area], LA[Playground,Running Track], Chicago[Tennis Court,Playground]. Amenities compared as an unordered pair.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 248 OUT1 360
✓	FindUpcomingEventsInThreeCitiesBFCL/parallel_194: three event_finder_find_upcoming calls — jazz/LA(14d), rock/Chicago(10d), classical/Boston(7d). All day windows explicitly stated -> days_ahead required (7 for Boston, which also equals the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 261 OUT1 095
✓	GetCurrentWeatherForFourCitiesBFCL/parallel_199: four get_current_weather calls — New York, Los Angeles, London, Tokyo, with both temperature and humidity included. Prompt explicitly asks to include both -> flags == true (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 343 OUT1 986
✓	GetDetailedBriefsForTwoMoviesBFCL/parallel_195: two movie_details_brief calls — Inception and The Dark Knight, extra_info true. Prompt asks for Director/Cast/Awards -> extra_info == true required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 027 OUT1 797
✓	GetLawsuitDetailsForTwoCasesBFCL/parallel_196: two get_lawsuit_details calls — case 12345 (NY Supreme Court, with verdict) and case 67890 (LA Superior Court, no verdict). "would like the verdict" -> true; "do not need" -> null/false.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 176 OUT2 792
✓	GetLawsuitInfoForTwoCasesByYearBFCL/parallel_197: two lawsuit_info calls — 12345ABC (2018, New York) and 67890XYZ (2019, California). Years are explicitly stated -> required. First location "New York" may be omitted (matches the default).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 167 OUT1 427
✓	GetStockPricesForTwoCompaniesOverDaysBFCL/parallel_179: two get_stock_price (by-days variant) calls — Apple on NASDAQ for 10 days, Microsoft on NYSE for 15 days. Both exchanges are explicitly named -> required (NASDAQ / NYSE). The tool's 'company' field is a ticker symbol; company names are accepted as synonyms.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 050 OUT1 638
✓	SearchLawsuitsForTwoEntitiesInCaliforniaBFCL/parallel_198: two lawsuit_search calls — Google/Santa Clara and Facebook/San Mateo, state California. Prompt states "both in the state of California" -> state required (accept "California"/"CA").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 087 OUT1 056
✓	SendTwoEmailsWithCcAndBccBFCL/parallel_193: two send_email calls — Project Update to john.doe (cc manager, bcc hr) and Meeting Reminder to jane.doe (cc assistant, bcc hr). Bodies matched exactly.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 373 OUT793
Tool Calling - Parallel Calls: Statistics & Finance 12 passed1 failed	92.3%	IN125 260 OUT33 259
✓	CalculateBinomialProbabilitiesForBasketballShotsBFCL/parallel_163: three calc_binomial_prob calls — (10,6,0.6), (10,6,0.5), (15,6,0.5).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 157 OUT1 784
×	CalculateCompoundInterestForThreePeriodsBFCL/parallel_170: three calculate_compound_interest calls — principal 5000, rate 0.025, time 2/3/5. Prompt states "compounded quarterly, 4 times in a year" -> n == 4 required (tightened; also ground truth).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN16 731 OUT4 609
✓	CalculateDiscountedCashFlowForTwoBondsBFCL/parallel_169: two calculate_discounted_cash_flow calls — (50,5,0.05) and (60,7,0.04). Prompt explicitly states "face value of the bond is $1000" for both -> face_value == 1000 required (ground truth also accepts omission via the default, but an explicitly-stated value must be supplied).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 994 OUT2 138
✓	CalculateEventProbabilitiesForCardDeckBFCL/parallel_162: three probability_of_event calls — Ace 4/52 as decimal, heart 13/52 as decimal, red card 26/52 as ratio. format_as_ratio: "as a decimal" -> false or omitted; "as a ratio" -> true (required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 124 OUT1 854
✓	CalculateQuarterlyDividendForThreeScenariosBFCL/parallel_168: three finance_calculate_quarterly_dividend_per_share calls — (5M,2M), (6M,2M), (6M,2.5M).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 180 OUT1 862
✓	CalculateReturnOnEquityForTwoCompaniesBFCL/parallel_171: two calculate_return_on_equity calls — (1M,5M,200k) and (2M,10M, no dividends). Second company "did not pay any dividends" -> dividends_paid null or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 955 OUT1 718
✓	CalculateRoiForThreeStocksBFCL/parallel_174: three calculate_return_on_investment calls — (150,180,20), (200,210,30), (250,300,40). Dividends are explicitly stated per stock -> required.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 096 OUT1 799
✓	ComputeRSquaredForTwoRegressionModelsBFCL/parallel_167: two linear_regression_get_r_squared calls on the same CSV — swap education/credit_score as dependent variable. Independent-variable order follows the prompt/ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 950 OUT1 370
✓	PredictFutureValueForTwoInvestmentsBFCL/parallel_172: two finance_predict_future_value calls — (5000,0.05,10) and (7000,0.04,15). Prompt states "compound interest annually" -> compounding_periods_per_year == 1 required (tightened; also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 964 OUT1 623
✓	PredictHousePricesForThreeHousesBFCL/parallel_166: three predict_house_price calls — NY 2500/3/2000, LA 3000/3/2005, Chicago 2000/2/1995. The second house is described as "similar" with only area/year/location changed, so rooms == 3 per ground truth.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 147 OUT4 779
✓	PredictProfitForTwoFundsBFCL/parallel_173: two investment_predictProfit calls — (5000,0.07,5) and (8000,0.05,7).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 916 OUT3 039
✓	RunChiSquaredTestForTwoContingencyTablesBFCL/parallel_164: two chi_squared_test calls — tables [45,55,35,65] and [30,70,50,50], alpha 0.05. Prompt states "significance level of 0.05 for both" -> alpha == 0.05 required (tightened). Table order is fixed.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 900 OUT2 628
✓	RunTTestAtTwoSignificanceLevelsBFCL/parallel_165: two t_test calls — same datasets, alpha 0.05 then 0.01 (both stated -> required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 146 OUT4 056
Tool Calling - Simple: Arts, Venues & Music (set 1) 7 passed1 failed	87.5%	IN50 785 OUT11 994
✓	SimpleConcertDetailsBeyonceBFCL/simple_python_286 (concert.get_details): artist=Beyonce, location=San Diego, date=April 2022.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 358 OUT1 264
×	SimpleConcertInfoPopNyBFCL/simple_python_284 (concert_info.get): location=New York, date=next month, genre=Pop.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: (args.date?.ToLower()).AnyOf(["next month", "2023-06-01", "06/01/2023", "jun.1,2023", "june 2023"])	Failed	IN6 333 OUT3 093
✓	SimpleGenerateMelodyC4BFCL/simple_python_291 (music_generator.generate_melody): key=C, start_note=C4, length=16, tempo=120 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 381 OUT1 220
✓	SimpleKeySignatureCsharpBFCL/simple_python_298 (music_theory.key_signature): key='C#', scale_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 318 OUT776
✓	SimpleMuseumHoursLouvreBFCL/simple_python_273 (museum.get_hours): name='Louvre Museum', location=Paris, day omitted or Monday.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 325 OUT1 761
✓	SimplePaintNeededWall30x12BFCL/simple_python_259 (calculate_paint_needed): coverage_rate=400, length=30, height=12.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL	Passed	IN6 381 OUT889
✓	SimpleReligionHistoryChristianity16cBFCL/simple_python_252 (get_religion_history): religion=Christianity, century=16, sort_by=importance, count=10.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 358 OUT1 257
✓	SimpleSculptureInfoPlensaBFCL/simple_python_263 (get_sculpture_info): artist_name='James Plensa', detail=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 331 OUT1 734
Tool Calling - Simple: Arts, Venues & Music (set 2) 7 passed1 failed	87.5%	IN57 369 OUT19 414
✓	SimpleComposeMelodyCFGBFCL/simple_python_292 (compose_melody): progression=[C,F,G], measures=4, instrument=Piano or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 321 OUT519
✓	SimpleConcertFindDetailsWeekndBFCL/simple_python_290 (concert.find_details): artist=The Weeknd, month=December, year=2022 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 297 OUT5 990
✓	SimpleFindConcertRockChicagoBFCL/simple_python_285 (find_concert): location=Chicago, price=100, genre=Rock.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 307 OUT1 191
✓	SimpleMuseumInfoMetBFCL/simple_python_274 (museum_info): museum_name=Metropolitan Museum of Art, info_type=opening_hours or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 295 OUT1 649
✓	SimpleMusicalScaleCsharpBFCL/simple_python_299 (musical_scale): key='C#'/'C sharp', scale_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 286 OUT1 835
✓	SimplePaintRequirementWallExclusionBFCL/simple_python_260 (paint_requirement.calculate): area {width=20,height=12}, paint_coverage=350, exclusion {type=window, area=15}.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN6 416 OUT2 433
✓	SimpleReligionHistoryChristianity300To400BFCL/simple_python_254 (get_religion_history): religion=Christianity, start_year=300, end_year=400, event_type=all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 322 OUT2 086
×	SimpleSculptureDetailsDavidBFCL/simple_python_264 (sculpture.get_details): artist=Michelangelo, title=David, detail=size.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3	Failed	IN13 125 OUT3 711
Tool Calling - Simple: Arts, Venues & Music (set 3) 9 passed	100.0%	IN60 208 OUT15 679
✓	SimpleChordProgressionKeyBFCL/simple_python_297 (music.theory.chordProgression): progression=[I,V,vi,IV], bools any.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 693 OUT1 056
✓	SimpleCompoundInterest10kBFCL/simple_python_269 (calculate_compound_interest): principle=10000, interest_rate=0.05, time=10, compounds_per_year=1 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 736 OUT2 021
✓	SimpleConcertSearchClassicalLaBFCL/simple_python_287 (concert.search): genre=classical, location=Los Angeles/LA, date='this weekend', price_range=cheap.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 692 OUT2 579
✓	SimpleCreateMixCMajorBFCL/simple_python_293 (music_composer.create_mix): scale='C Major', note_duration=quarter, track_length=180.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 715 OUT1 128
✓	SimpleGenerateCircleImageRedBFCL/simple_python_256 (generate_circle_image): radius=50, color=Red, background=white or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 684 OUT565
✓	SimpleMuseumWorkingHoursLouvreBFCL/simple_python_276 (museum_working_hours.get): museum=Louvre, location=Paris, day=Monday or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 670 OUT2 081
✓	SimpleRetrieveReligionInfoBuddhismBFCL/simple_python_253 (retrieve_religion_info): religion_name=Buddhism, detail_level=full.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 665 OUT2 077
✓	SimpleSculptureSearchChicago19cBFCL/simple_python_265 (sculpture_search): location=Chicago, time_frame='19th century', material=all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 682 OUT2 293
✓	SimpleTimeDifferenceSfSydneyBFCL/simple_python_250 (get_time_difference): place1=San Francisco/SF, place2=Sydney.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL	Passed	IN6 671 OUT1 879

Tool Calling - Irrelevance: Architecture & Artwork 7 passed2 failed

77.8%

IN38 099 OUT7 570

✓

NoToolForApartmentRentCostBFCL/irrelevance_158: apartment rental cost asked; only a new-construction cost estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 461 OUT382

×

NoToolForBestJazzSingerBFCL/irrelevance_164: best Jazz singer asked; only a top-instrument-players search (singers are not instrument players) is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 933 OUT1 822

✓

NoToolForCathedralCeilingDesignBFCL/irrelevance_157: how to design a cathedral ceiling asked; only a lookup of facts about an existing building is available (retrieves data, does not advise on design) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 457 OUT687

✓

NoToolForHolyBookDefinitionBFCL/irrelevance_144: definition of a holy book asked; only a search-within-a-named-holy-book tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 455 OUT448

✓

NoToolForLectureTeacherBFCL/irrelevance_167: teacher of upcoming lectures asked; only an upcoming-concert details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 458 OUT313

✓

NoToolForMostFamousSculptureBFCL/irrelevance_153: most famous sculpture asked; only a sculpting-tools lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 463 OUT892

✓

NoToolForPianoMaintenanceTipsBFCL/irrelevance_166: piano maintenance tips asked; only an instrument rental-prices lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 458 OUT951

×

NoToolForScreamArtistBFCL/irrelevance_159: artist behind 'The Scream' asked. NOTE: the only tool (artwork_search) requires a museum_location that the user did not provide, so it cannot be correctly called; BFCL labels this irrelevant. Kept faithfully with ground truth = no tool call (flagged as borderline).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 951 OUT995

✓

NoToolForSculptureRockTypeBFCL/irrelevance_152: which rock suits a garden sculpture asked; only a sculpture 3D-model creator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 463 OUT1 080

Tool Calling - Irrelevance: Buildings, Sculpture & Music 7 passed1 skipped

100.0%

IN22 989 OUT7 683

–

NoToolForCelloInstrumentTypeBFCL/irrelevance_165: instrument classification of a cello asked. NOTE: BFCL labels this irrelevant, but the only tool (get_instrument_info with detail=type) arguably answers it; flagged for review as a borderline BFCL item. Kept faithfully with ground truth = no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_instrument_info' with detail='type' answers 'what type of instrument is a cello' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForDeclarationSignersBFCL/irrelevance_137: who signed the declaration asked; only a historical-event DATE lookup is available (returns a date, not the signatories) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 282 OUT673

✓

NoToolForEiffelTowerHeightBFCL/irrelevance_156: Eiffel Tower height asked; only an architecture-plan generator (creates a new design, does not retrieve facts) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 286 OUT582

✓

NoToolForFamousDancersBFCL/irrelevance_162: famous dancers asked; only a museum-visual-artists lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 288 OUT1 491

✓

NoToolForReformationInitiatorBFCL/irrelevance_145: who initiated the Reformation asked; only a religious-event YEAR lookup is available (returns a year, not a person) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 281 OUT905

✓

NoToolForScientistDefinitionBFCL/irrelevance_143: definition of "scientist" asked; only a named-historical-figure info lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 278 OUT695

✓

NoToolForSellingGuitarBFCL/irrelevance_163: how to sell a guitar asked; only an instrument-tuning tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 283 OUT814

✓

NoToolForSimilarBlueColorBFCL/irrelevance_148: a color similar to blue asked; only a complementary-color tool (opposite, not similar) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 291 OUT2 523

Tool Calling - Irrelevance: Cell Biology & Genetics 7 passed1 failed2 skipped

87.5%

IN34 134 OUT6 603

×

NoToolForClottingBloodCellsBFCL/irrelevance_32: blood cell type responsible for clotting asked; only a nucleus-count-based cell-type classifier is available.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN7 609 OUT1 200

✓

NoToolForCommonAncestorBFCL/irrelevance_41: common ancestor of two animals asked (phylogenetics knowledge); only a fibonacci-sequence calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 789 OUT358

✓

NoToolForEvolutionaryAlgorithmInGameAIBFCL/irrelevance_43: how to apply an Evolutionary Algorithm in game AI asked (advice); only a creature-evolution simulator is available (wrong context).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 790 OUT1 210

✓

NoToolForGeneVariantFrequencyBFCL/irrelevance_36: gene-variant frequency in a population asked; only a dominant-trait-from-alleles tool is available (the true-match variant-frequency tool is deliberately isolated in A4).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 795 OUT802

–

NoToolForIdentifyingGeneticSequenceBFCL/irrelevance_33: interpret a genetic code sequence asked; only a species-identification-from-sequence tool is available (returns a species, not what the sequence means).0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'identify_species' (identifies a species from a genetic sequence) plausibly answers the query about a DNA sequence, so a competent model may call it - BFCL-mislabeled/borderline-relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForIncreasingDeerPopulationBFCL/irrelevance_39: how-to advice on growing an animal population asked; only a population-growth CALCULATOR (needs numeric rates) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 789 OUT869

✓

NoToolForLionMatingProcessBFCL/irrelevance_35: mating process of a species asked; only a dominant/recessive genetic-traits lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 786 OUT672

✓

NoToolForMitochondriaCountBFCL/irrelevance_29: number/count of mitochondria asked; only a cell-part FUNCTION lookup is available (wrong query direction).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 790 OUT863

✓

NoToolForPandaEvolutionaryHistoryBFCL/irrelevance_42: evolutionary history asked (a narrative fact); only a biodiversity-index calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 786 OUT629

–

NoToolForWhaleGeneSequenceBFCL/irrelevance_44: the actual gene sequence of a species asked; only a gene-sequence GENERATOR (needs numeric mutation rate + duration) is available.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'gene_sequencer' (generate gene sequences for evolutionary changes) directly answers the query - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Irrelevance: Color, Geometry & Population 11 passed

100.0%

IN40 836 OUT6 454

✓

NoToolForColorInComputerVisionBFCL/irrelevance_131: color meaning in computer vision asked; only a color-in-personality-psychology analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 712 OUT873

✓

NoToolForDeclarationSigningDateBFCL/irrelevance_135: signing date asked; only date-arithmetic (add days) and age tools are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 710 OUT427

✓

NoToolForElectricityDiscovererBFCL/irrelevance_140: who discovered electricity asked; only an electric-current calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 706 OUT627

✓

NoToolForHexColorMixBFCL/irrelevance_146: mixing two hex colors asked; only a prophet-details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 722 OUT812

✓

NoToolForIndependenceSigningDateBFCL/irrelevance_138: signing date asked; only an age calculator (needs birthdate + current date) and a date-adder are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 710 OUT433

✓

NoToolForMostVisitedMarketBFCL/irrelevance_161: most visited market asked; only a museum visitation-statistics lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 712 OUT714

✓

NoToolForNbaFinalWinnerBFCL/irrelevance_133: NBA final winner asked; only a historical-battle details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 713 OUT227

✓

NoToolForPicassoPaintingYearBFCL/irrelevance_151: year of a Picasso painting asked; only a sculpture-dimensions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 711 OUT556

✓

NoToolForRelativityAuthorBFCL/irrelevance_142: who proposed special relativity asked; only a chemical-element properties lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 715 OUT248

✓

NoToolForSculptureCreatorBFCL/irrelevance_155: sculpture creator asked; only a max-seating-capacity calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 711 OUT910

✓

NoToolForWhichColorsToMixBFCL/irrelevance_150: which colors to mix for a shade asked; only an RGB-to-Pantone converter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 714 OUT627

Tool Calling - Irrelevance: Cooking, Currency & Markets 6 passed2 failed

75.0%

IN32 473 OUT8 295

×

NoToolForBakerySourdoughBFCL/irrelevance_223: a BAKERY selling sourdough asked; only a grocery-shop product finder available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 506 OUT1 582

✓

NoToolForChickenRecipeBFCL/irrelevance_211: the RECIPE to cook chicken asked; only a cook-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 241 OUT834

✓

NoToolForChocolateCakeIngredientsBFCL/irrelevance_220: cake INGREDIENTS asked; only a cooking-TEMPERATURE tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 244 OUT1 044

✓

NoToolForCurrencyToGbpBFCL/irrelevance_233: CURRENCY conversion asked; only a WEATHER tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 246 OUT440

×

NoToolForHowToBoilAnEggBFCL/irrelevance_212: best WAY to boil an egg asked; only a boiling-TIME calculator available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 512 OUT2 066

✓

NoToolForLasagnaRecipeBFCL/irrelevance_219: the RECIPE for lasagna asked; only a recipe-CALORIE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 241 OUT1 232

✓

NoToolForLegExercisesBFCL/irrelevance_221: leg EXERCISES asked; only a grocery-list generator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 241 OUT475

✓

NoToolForPetStoreBFCL/irrelevance_224: a PET store asked; only a GROCERY-store locator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 242 OUT622

Tool Calling - Irrelevance: Dates & Elements 9 passed1 failed

90.0%

IN39 945 OUT9 546

✓

NoToolForHydrogenPropertiesBFCL/irrelevance_141: chemical properties of Hydrogen asked; only a scientist-contributions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 626 OUT1 560

✓

NoToolForLargestPlanetBFCL/irrelevance_139: largest planet asked; only a star-info lookup (needs a star name) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 626 OUT1 176

✓

NoToolForMostImportantProphetBFCL/irrelevance_147: most important prophet asked; only a two-color mixer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 626 OUT363

×

NoToolForMuseumVisitFrequencyBFCL/irrelevance_160: museum member visit frequency asked; only a single most-frequent-visitor lookup is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN7 306 OUT3 001

✓

NoToolForPantoneOfSkyBlueBFCL/irrelevance_149: Pantone code of sky blue asked; only a paint-proportions calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 628 OUT1 197

✓

NoToolForPersonalityTypeBFCL/irrelevance_130: user asks what personality type they are; only a big-five calculator (which needs the user's rated answers) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 624 OUT493

✓

NoToolForStadiumSeatingCapacityBFCL/irrelevance_154: stadium seating capacity asked; only a sculpture-creator lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 627 OUT470

✓

NoToolForVicePresidentBFCL/irrelevance_136: Vice President asked; only a US-president-by-year lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 626 OUT408

✓

NoToolForWaterlooCasualtiesBFCL/irrelevance_132: casualty count of a battle asked; only a historical-event date lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 628 OUT685

✓

NoToolForWorldCupWinnerBFCL/irrelevance_134: sports World Cup winner asked; only a historical-battle outcome predictor is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 628 OUT193

Tool Calling - Irrelevance: Electromagnetism, Thermo & Data 10 passed

100.0%

IN34 045 OUT6 124

✓

NoToolForElectronRestMassBFCL/irrelevance_23: mass of an electron asked (a knowledge fact); only magnetic-field / current tools are available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 397 OUT179

✓

NoToolForFlightDurationBFCL/irrelevance_20: flight duration asked; only a currency converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 400 OUT323

✓

NoToolForFreezingPointBFCL/irrelevance_25: freezing point asked; only boiling-point calculators are available (wrong phase transition).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 407 OUT830

✓

NoToolForGasAmountGeneratedBFCL/irrelevance_26: amount of gas generated asked; only a gas-PRESSURE calculator is available (wrong quantity).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 420 OUT1 122

✓

NoToolForGiniCoefficientBFCL/irrelevance_48: Gini coefficient asked; only a GDP-fetch tool is available (different economic indicator).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 400 OUT498

✓

NoToolForHeatingWaterBFCL/irrelevance_27: heat energy to warm water asked. NOTE: the original tool (calculate_heat) is a fair match for this query (only the well-known specific heat is missing), which would make "no call" ambiguous; per human-baseline fairness it is deliberately NOT shown here (kept as a distractor in A3), so no visible tool fits and "no call" is unambiguously correct.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 411 OUT1 948

✓

NoToolForHexagonSidesBFCL/irrelevance_28: number of sides of a hexagon asked (basic geometry knowledge); only a boiling-point tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 398 OUT179

✓

NoToolForSymbolicMagneticFieldBFCL/irrelevance_22: magnetic field asked with SYMBOLIC values (distance 'r', current 'I'); the two magnetic-field tools require concrete numeric distance + current, so there is nothing to pass -> no fit.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 411 OUT457

✓

NoToolForTimeDifferenceBFCL/irrelevance_49: time-zone difference asked; only a geo-coordinate lookup tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 400 OUT384

✓

NoToolForWorldSeriesWinnerBFCL/irrelevance_53: sports trivia asked; only a SQL database-query runner is available (no database/query provided).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 401 OUT204

Tool Calling - Irrelevance: Entertainment, Sports & Knowledge 9 passed

100.0%

IN31 410 OUT3 257

✓

NoToolForBasketballResultBFCL/irrelevance_84: basketball game result asked; only a stock-price retrieval tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 490 OUT272

✓

NoToolForBestSoccerPlayerBFCL/irrelevance_79: best current soccer player asked; only a stock-market prediction tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 488 OUT970

✓

NoToolForCaliforniaGovernorBFCL/irrelevance_87: current Governor of California asked; only a court-case filing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 486 OUT239

✓

NoToolForCrimeThrillerMovieListBFCL/irrelevance_88: best crime-thriller movies of 2020 asked; only a document forgery detector is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 495 OUT266

✓

NoToolForFifaWorldCupWinnerBFCL/irrelevance_80: FIFA World Cup 2010 winner asked; only a stock-ticker lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 491 OUT174

✓

NoToolForFrenchTranslationBFCL/irrelevance_61: French translation of a word asked; only an SVM-model tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 490 OUT216

✓

NoToolForHorrorMovieListBFCL/irrelevance_77: horror movie recommendations asked; only an investment future-value tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 489 OUT738

✓

NoToolForPresidentialElectionWinnerBFCL/irrelevance_85: 2020 presidential election winner asked; only a criminal-case detail lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 491 OUT181

✓

NoToolForSportsNewsBFCL/irrelevance_76: latest sports news asked; only a portfolio investment-trend analyzer is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 490 OUT201

Tool Calling - Irrelevance: Finance & Economics 7 passed1 failed

87.5%

IN30 189 OUT4 164

✓

NoToolForAverageHousePriceBFCL/irrelevance_69: market-wide average house price asked; only a regression predictor (needs a trained model + feature vector, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 346 OUT345

✓

NoToolForCompoundInterestBFCL/irrelevance_70: compound-interest computation asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 372 OUT976

✓

NoToolForGoldPriceBFCL/irrelevance_78: today's gold spot price asked; only a bond-price calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 348 OUT199

✓

NoToolForHorrorMovieCatalogBFCL/irrelevance_81: horror movie recommendations asked; only a historical stock-price fetcher is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 349 OUT434

×

NoToolForLossProjectionBFCL/irrelevance_73: a loss projection asked; only a revenue-prediction tool is available (revenue != loss).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 714 OUT1 136

✓

NoToolForPoundToKilogramBFCL/irrelevance_75: a mass unit conversion asked; only a portfolio-returns calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 347 OUT167

✓

NoToolForProfitMarginBFCL/irrelevance_71: profit margin asked; only an ROI calculator is available (ROI != margin) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 369 OUT556

✓

NoToolForStockNewsBFCL/irrelevance_82: stock news asked; only a capital-gains calculator is available (gains != news) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 344 OUT351

Tool Calling - Irrelevance: Finance Near-Matches 8 passed1 failed

88.9%

IN34 411 OUT12 102

✓

NoToolForBitcoinPriceBFCL/irrelevance_56: current bitcoin price asked; only a generic database record-select tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 426 OUT772

✓

NoToolForBookstoreLookupBFCL/irrelevance_55: which online bookstore sells a title asked; only an add-to-cart tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 431 OUT470

✓

NoToolForBusinessRateOfReturnBFCL/irrelevance_74: rate of return asked; only a net-profit calculator is available (profit != rate of return).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 447 OUT1 173

✓

NoToolForCoinFlipProbabilityBFCL/irrelevance_66: probability of a coin landing heads asked; only a standard-deviation tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 431 OUT434

✓

NoToolForCurrentInterestRateBFCL/irrelevance_83: the current market interest rate asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 425 OUT2 816

×

NoToolForExternalRateOfReturnBFCL/irrelevance_72: EXTERNAL rate of return asked; only an internal-rate-of-return (IRR) tool is available (ERR != IRR, even though cash flows are given) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 945 OUT4 236

✓

NoToolForFinalPriceWithoutBasePriceBFCL/irrelevance_60: final price after discount+tax asked, but the original price is never given, so the (domain-matching) calculateFinalPrice cannot be correctly called -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 445 OUT280

✓

NoToolForFrontendDevelopmentBFCL/irrelevance_62: how to build an e-commerce frontend asked; only a recommender-model builder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 431 OUT486

✓

NoToolForHighestGrossingMovieBFCL/irrelevance_54: highest-grossing movie of all time asked; only a movie search (requires a specific title + year, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 430 OUT1 435

Tool Calling - Irrelevance: Food, Time & Distance 2 passed2 failed1 skipped

50.0%

IN16 296 OUT5 543

×

NoToolForCakeBakingMachineBFCL/irrelevance_218: a baking MACHINE/appliance asked; only a recipe retriever available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN5 450 OUT2 462

✓

NoToolForDrivingTravelTimeBFCL/irrelevance_215: driving TRAVEL TIME asked; only a restaurant-bill tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 710 OUT781

×

NoToolForLondonTimezoneNameBFCL/irrelevance_226: the NAME of London's timezone asked; only a local-TIME-by-timezone tool available (it needs the timezone as input and returns a time, not the timezone name) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN5 422 OUT1 213

–

NoToolForPizzaInBostonBFCL/irrelevance_213: a place for pizza asked; a cuisine+location restaurant_finder is available. REVIEW FLAG (fairness): restaurant_finder(cuisine=pizza/Italian, location=Boston) plausibly answers this, so a competent human might legitimately call it. BFCL labels it irrelevant; kept as such and marked Hard because the intended "no call" is debatable. No OTHER query here is a restaurant search, so the tool cannot wrongly satisfy the file's other cases.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'restaurant_finder' (cuisine+location) answers 'good place for pizza in Boston' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForTriangleHypotenuseBFCL/irrelevance_238: the HYPOTENUSE of a triangle asked; only a geo lat/long distance calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 714 OUT1 087

Tool Calling - Irrelevance: Geo, Travel & Sentiment 7 passed1 failed1 skipped

87.5%

IN30 713 OUT5 087

✓

NoToolForAverageTemperatureBmiBFCL/irrelevance_110: an average December temperature asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 396 OUT558

✓

NoToolForCaseLawyerBFCL/irrelevance_95: the lawyer of a case asked; only a case-outcome retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 399 OUT752

✓

NoToolForCurrentTimeSunriseBFCL/irrelevance_102: the current time asked; only a sunrise/sunset time calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 395 OUT299

✓

NoToolForFamousQuoteBFCL/irrelevance_119: a famous quote asked; only a sentiment-analysis tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 393 OUT138

✓

NoToolForFastestRouteBFCL/irrelevance_114: the fastest route asked; only a closest-airport finder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 397 OUT574

–

NoToolForRestaurantsInLondonBFCL/irrelevance_113: restaurants in London asked; only a generic points-of-interest locator is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'find_pois' (locate points of interest by category) answers 'restaurants in London' (restaurants are POIs) - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForSportsBrainImpactBFCL/irrelevance_120: the neurological impact of sports asked; only a caffeine neurological-impact tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 396 OUT688

✓

NoToolForTrafficConditionBFCL/irrelevance_112: the current traffic condition asked; only an address geocoder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 395 OUT274

×

NoToolForTravelTimeByCarBFCL/irrelevance_115: travel time by car asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2

Failed

IN6 942 OUT1 804

Tool Calling - Irrelevance: Legal & Case Lookup 9 passed1 failed

90.0%

IN39 074 OUT4 620

✓

NoToolForAppleCaloriesBFCL/irrelevance_100: calories in an apple asked; only a litigation-cost calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 548 OUT176

✓

NoToolForBrainWeightBFCL/irrelevance_122: average human brain weight asked; only a per-patient neural-activity tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 550 OUT232

✓

NoToolForCaseJudgementBFCL/irrelevance_93: a case judgement asked; only a law-firm impactful-cases lister is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 548 OUT396

✓

NoToolForChineseRestaurantBFCL/irrelevance_97: a Chinese-restaurant recommendation asked; only a lawsuit-filing tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 551 OUT340

✓

NoToolForMovieRecommendationBFCL/irrelevance_116: a movie recommendation asked; only a word-count tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 549 OUT737

×

NoToolForNbaChampionBFCL/irrelevance_118: the 1996 NBA champion asked; only an NBA playoff-brackets display tool is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN7 118 OUT955

✓

NoToolForPaintingEiffelSettlementBFCL/irrelevance_98: how long to paint the Eiffel Tower asked; only a lawsuit-settlement estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 552 OUT283

✓

NoToolForSecurityGuardSearchBFCL/irrelevance_92: finding a security guard asked; only a lawyer-search tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 558 OUT418

✓

NoToolForSphereVolumeBFCL/irrelevance_104: sphere volume asked; only a circular-park area tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 552 OUT752

✓

NoToolForTrafficLawsBFCL/irrelevance_99: traffic laws asked; only a lawsuit-search tool is available (lawsuits, not laws) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 548 OUT331

Tool Calling - Irrelevance: Math & Science Distractors 8 passed

100.0%

IN25 236 OUT5 547

✓

NoToolForDefiniteIntegralBFCL/irrelevance_9: definite integral asked; only an income-tax calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 157 OUT460

✓

NoToolForIntegralOfCubeBFCL/irrelevance_8: definite integral asked; only a string-to-int converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 150 OUT1 212

✓

NoToolForLatestMoviesBFCL/irrelevance_15: latest movie releases asked; only a physics final-velocity tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 144 OUT247

✓

NoToolForPerpendicularSlopeBFCL/irrelevance_3: slope of a perpendicular line asked; only a critical-points finder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 162 OUT1 155

✓

NoToolForPrimeFactorsBFCL/irrelevance_13: prime factorization asked; only a compound-interest calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 147 OUT341

✓

NoToolForRectangleAreaBFCL/irrelevance_6: rectangle area asked; only a circle-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 158 OUT227

✓

NoToolForTriangleAreaBFCL/irrelevance_0: triangle area asked; only a BMI tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 158 OUT398

✓

NoToolForUpwardAccelerationBFCL/irrelevance_14: acceleration asked (it is just gravity); the tempting max-height and final-velocity tools compute neither the acceleration nor what is asked -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 160 OUT1 507

Tool Calling - Irrelevance: Math & Statistics 5 passed1 failed

83.3%

IN21 155 OUT7 989

✓

NoToolForBinomialMeanBFCL/irrelevance_67: the MEAN of a binomial experiment asked; only p-value tools are available (they return a p-value, not the mean) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 026 OUT2 627

✓

NoToolForCivilLawComponentsBFCL/irrelevance_90: the components/categories of Civil Law asked; only a noise-complaint filing tool is available (filing a complaint does not explain law) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN2 998 OUT661

✓

NoToolForCoinTossMaxHeadsBFCL/irrelevance_63: how many heads CAN be obtained from tossing 3 coins (a count/maximum, not a probability) asked; only a probability-of-an-event calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 002 OUT366

×

NoToolForNullHypothesisStdDevBFCL/irrelevance_68: the STANDARD DEVIATION of a null-hypothesis test asked; only p-value tools are available (they return a p-value, not a standard deviation) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 121 OUT1 955

✓

NoToolForPowerOfTwoBFCL/irrelevance_59: raising a number to a power asked; only a linear-equation solver is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 001 OUT1 356

✓

NoToolForQuadraticEquationBFCL/irrelevance_57: solving a quadratic equation asked; only a genetic-algorithm optimizer and a LINEAR equation solver are available (a quadratic is not linear) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 007 OUT1 024

Tool Calling - Irrelevance: Medical, Social & Personality 6 passed3 skipped

100.0%

IN20 452 OUT4 664

✓

NoToolForBigMacCaloriesBFCL/irrelevance_123: calories of a Big Mac asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 407 OUT461

✓

NoToolForEconomicHappinessEffectBFCL/irrelevance_126: the effect of economic status on happiness asked; only a happiness-index lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 409 OUT2 239

✓

NoToolForFriendMbtiBFCL/irrelevance_129: a friend's MBTI asked; only a personality-type evaluator (needs trait percentages) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 407 OUT269

–

NoToolForLatestTechTrendBFCL/irrelevance_124: the latest technology trend asked; only a trending-topics-by-category tool is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_social_trends' (trending topics by category) answers 'latest trend in technology' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForMajorAirportsBFCL/irrelevance_117: major US airports asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 410 OUT701

–

NoToolForMotorNeuronDiseaseBFCL/irrelevance_121: motor neuron disease info asked; only a generic disease-info retriever is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'medical_records_get_disease_info' directly answers 'information on motor neuron diseases' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForPopularBooksBFCL/irrelevance_125: popular books by an author asked; only a recent-tweets retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 409 OUT458

–

NoToolForTwitterMoodBFCL/irrelevance_127: overall twitter mood asked; only a twitter sentiment-analysis tool is available -> BFCL ground truth is no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'sentiment_analysis_twitter' directly answers 'general mood of twitter regarding the new iPhone' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForVegetableServingsBFCL/irrelevance_128: daily vegetable servings asked; only a personality-test scorer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 410 OUT536

Tool Calling - Irrelevance: Mixed Distractors 8 passed

100.0%

IN25 992 OUT4 608

✓

NoToolForAirQualityBFCL/irrelevance_40: current air quality asked; only a plant-biomass calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 250 OUT330

✓

NoToolForFavoriteRestaurantBFCL/irrelevance_31: a favorite restaurant recommendation asked; only a cell-division simulator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 246 OUT300

✓

NoToolForGlobalWarmingDocumentaryBFCL/irrelevance_38: a documentary (video) asked; only a scientific-research-paper retriever is available (wrong medium).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 247 OUT965

✓

NoToolForLasagnaIngredientsBFCL/irrelevance_47: recipe ingredients asked; only a flight-timetable tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 247 OUT495

✓

NoToolForLionDominantTraitBFCL/irrelevance_34: dominant genetic trait of a species asked; only a gene-variant-FREQUENCY lookup is available (different genetic output).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 249 OUT1 155

✓

NoToolForMultinucleatedCellNameBFCL/irrelevance_30: name of a cell type with multiple nuclei asked; only a blood-cell classifier (by shape/size) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 253 OUT478

✓

NoToolForPicnicSpotBFCL/irrelevance_37: a picnic spot asked; only a local-fauna information tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 246 OUT654

✓

NoToolForTranslationBFCL/irrelevance_52: natural-language translation asked; only a file-compression tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 254 OUT231

Tool Calling - Irrelevance: Motion, Electricity & Misc 10 passed

100.0%

IN35 408 OUT6 308

✓

NoToolForCarProjectileRangeBFCL/irrelevance_16: projectile-style "how far will a car travel" asked with symbolic v/t/theta; the range tool needs concrete numeric velocity + angle -> nothing to pass -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 551 OUT696

✓

NoToolForConeVolumeBFCL/irrelevance_19: cone volume asked; only an investment-return calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 545 OUT650

✓

NoToolForCurrentClockTimeBFCL/irrelevance_17: current clock time asked; only a travel-time (distance/speed) calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 534 OUT207

✓

NoToolForElectronMassBFCL/irrelevance_24: mass of an electron asked (a knowledge fact); only an electric-current tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 535 OUT302

✓

NoToolForFibonacciOf20BFCL/irrelevance_51: fibonacci number asked; only a cryptocurrency-price tool is available (the fibonacci calculator lives in a different file so it cannot satisfy this query).1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 536 OUT421

✓

NoToolForForceAngleFromMomentumBFCL/irrelevance_18: angle of a force from momentum asked; only a vector-angle-from-XY-components tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 541 OUT1 299

✓

NoToolForHorrorMovieListBFCL/irrelevance_50: movie recommendations asked; only a Celsius-to-Fahrenheit converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 541 OUT742

✓

NoToolForMagneticFieldNearWireBFCL/irrelevance_21: magnetic field near a current-carrying wire asked; only an EM-wave-amplitude tool is available (magnetic-field calculators are deliberately kept in a different file).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 550 OUT1 194

✓

NoToolForSineOf45DegreesBFCL/irrelevance_45: sine of an angle asked; only a polygon-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 536 OUT364

✓

NoToolForTeslaPriceBFCL/irrelevance_46: product price of a car asked; only a currency-exchange-rate tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 539 OUT433

Tool Calling - Irrelevance: Music Theory & Lookup 8 passed

100.0%

IN25 572 OUT4 451

✓

NoToolForAncientGreekLawmakersBFCL/irrelevance_171: who established laws in Ancient Greece asked (history); only a musical-piece composer search is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 194 OUT558

✓

NoToolForBasketballGameAtVenueBFCL/irrelevance_169: who is playing a basketball game at a venue asked; only concert-locating tools exist.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 197 OUT360

✓

NoToolForBluesScalePatternBFCL/irrelevance_176: the interval pattern of a blues scale asked; only composer lookups are available (get_blues_scale lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 199 OUT832

✓

NoToolForDonQuixoteAuthorBFCL/irrelevance_172: author of the novel Don Quixote asked; only music-composition lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 192 OUT220

✓

NoToolForFamousUsComposersBFCL/irrelevance_170: most famous US composers asked (general knowledge, no specific piece); only a chord-progression creator + piece-specific composer lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 195 OUT1 002

✓

NoToolForGrammyBestAlbumBFCL/irrelevance_177: Grammy Award winner asked; only a song chord-progression lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 200 OUT606

✓

NoToolForPrimaryTriadsOfKeyBFCL/irrelevance_173: primary triads of a key asked; only a genre-common-chords finder is available (primary_triads lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 198 OUT571

✓

NoToolForUniversityClassAvailabilityBFCL/irrelevance_168: availability of a university class asked; only concert-availability / concert-search tools are available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 197 OUT302

Tool Calling - Irrelevance: Probability & Legal 5 passed

100.0%

IN13 997 OUT5 720

✓

NoToolForFaceCardProbabilityBFCL/irrelevance_64: probability of drawing a face card from a deck asked; only a COIN-toss heads probability tool is available (coins, not cards) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 796 OUT469

✓

NoToolForHomeElectricityNeedBFCL/irrelevance_58: home electricity demand (kWh) asked; only a solar-panel COUNT calculator is available (it sizes panels, not electricity need, and needs sunlight hours not given) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN2 799 OUT1 158

✓

NoToolForNoiseComplaintProcessBFCL/irrelevance_91: how/whether to report a noise complaint asked; only a law-category listing tool is available (it lists legal categories, it does not file or advise on complaints) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 796 OUT782

✓

NoToolForRedMarbleCountBFCL/irrelevance_65: number of red marbles asked; only a determine-population tool is available (it returns a total population from probability + sample size, not a sub-category count) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN2 811 OUT1 591

✓

NoToolForTenantRightsBFCL/irrelevance_89: explaining tenant rights asked; only a contract-generation tool is available (generating a contract does not explain rights) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN2 795 OUT1 720

Tool Calling - Irrelevance: Recipes & Cooking 8 passed1 skipped

100.0%

IN27 642 OUT5 867

✓

NoToolForBestGameCharacterBFCL/irrelevance_206: best game character vs a dragon asked; only a best-WEAPON finder available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 459 OUT1 853

✓

NoToolForBestMovie2020BFCL/irrelevance_235: best MOVIE of 2020 asked; only a currency-exchange calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 456 OUT338

✓

NoToolForBuildingAPcBFCL/irrelevance_209: how to build a PC asked; only a cooking-recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 451 OUT1 055

✓

NoToolForFamousPlaceInParisBFCL/irrelevance_210: most famous PLACE in Paris asked; only a restaurant-by-dish search available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 453 OUT346

✓

NoToolForGameShortestPathBFCL/irrelevance_208: shortest in-game path asked; only a taxi-fare calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 461 OUT417

✓

NoToolForInchToMeterBFCL/irrelevance_234: LENGTH conversion (inches to metres) asked; only a CURRENCY converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 452 OUT273

✓

NoToolForLowestGameScoreBFCL/irrelevance_207: LOWEST game score asked; only a HIGHEST-score retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 456 OUT801

✓

NoToolForNewYorkPopulationBFCL/irrelevance_225: current POPULATION of New York asked; only a time-zone converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 454 OUT784

–

NoToolForQuickestFlightLondonTokyoBFCL/irrelevance_236: quickest way to fly London->Tokyo asked; a flight-duration tool is available. REVIEW FLAG (fairness): get_flight_duration plausibly answers this ("quickest ... by plane" ~= quickest flight duration), so a competent human might legitimately call it. Kept as BFCL labels it (irrelevant), but this item is borderline. Marked Hard because the intended "no call" is debatable.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_flight_duration' (with flight_type non-stop) plausibly answers 'quickest way ... by plane' - BFCL-mislabeled/borderline-relevant. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Irrelevance: Restaurants & Travel 7 passed1 failed

87.5%

IN29 855 OUT4 147

✓

NoToolForBeijingSunsetBFCL/irrelevance_227: SUNSET time asked; only a SUNRISE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 313 OUT412

✓

NoToolForClosestItalianRestaurantBFCL/irrelevance_216: FIND the closest Italian restaurant asked; only a currency converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 312 OUT236

✓

NoToolForFindSushiRestaurantBFCL/irrelevance_214: FIND a sushi restaurant asked; only a tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 314 OUT209

✓

NoToolForNearestPharmacyBFCL/irrelevance_237: FIND the nearest pharmacy asked; only a route-to-a-known-location planner available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 314 OUT355

✓

NoToolForPetFriendlyTrainStationBFCL/irrelevance_230: a pet-friendly TRAIN STATION asked; only hotel search tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 313 OUT256

✓

NoToolForSushiRestaurantsTokyoBFCL/irrelevance_229: FIND sushi RESTAURANTS asked; only hotel-booking tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 314 OUT277

✓

NoToolForThaiRestaurantChicagoBFCL/irrelevance_231: FIND a Thai RESTAURANT asked; only a hotel-availability checker available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 315 OUT570

×

NoToolForYogaRetreatBFCL/irrelevance_232: a yoga RETREAT asked; only a HOTEL search tool available. REVIEW FLAG (fairness): a retreat is not a hotel, but hotel_search_find_hotels accepts arbitrary amenities (spa, vegetarian) and a location, so it is somewhat tempting. BFCL labels this irrelevant (retreat != hotel); kept as such and marked Hard because the boundary is borderline.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 660 OUT1 832

Tool Calling - Irrelevance: Sports & Games (set 1) 9 passed1 failed

90.0%

IN39 285 OUT9 061

✓

NoToolForAssistAverageBFCL/irrelevance_185: assist average asked; only an average-SCORING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 570 OUT1 167

✓

NoToolForBestChessMoveBFCL/irrelevance_194: best next move asked; only a board-game-STATISTICS tool is available (the move-suggesting board_analyser lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 572 OUT567

✓

NoToolForChessArrangementCountBFCL/irrelevance_196: total theoretical arrangements asked; only a final-score calculator is available (possible_moves lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 572 OUT733

✓

NoToolForFootballTeamRankingBFCL/irrelevance_186: ranking of a team asked; only a season-MVP tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 568 OUT897

✓

NoToolForIceCreamStoreBFCL/irrelevance_201: find a physical store asked; only a video-game guide is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 565 OUT319

×

NoToolForMostHeartsInPokerBFCL/irrelevance_199: who holds most of a suit asked; only a poker-dealing tool is available (get_highest_card_holder lives in E4) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN7 169 OUT1 966

✓

NoToolForPlayerBiographyBFCL/irrelevance_189: a biographical "who is X" asked; only a top-ranked-athlete lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 565 OUT818

✓

NoToolForTeamRosterBFCL/irrelevance_192: full roster asked; only a top-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 567 OUT1 221

✓

NoToolForUnoRulesBFCL/irrelevance_198: the rules of a game asked; only a deck-shuffling tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 569 OUT990

✓

NoToolForWorldCupWinnerBFCL/irrelevance_191: who won a tournament asked (no team named); only a per-team match-stats tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 568 OUT383

Tool Calling - Irrelevance: Sports & Games (set 2) 8 passed1 failed

88.9%

IN34 683 OUT8 564

✓

NoToolForBlackjackAceRuleBFCL/irrelevance_200: a rules question asked; only a poker highest-card-holder tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 467 OUT490

✓

NoToolForBookAuthorBFCL/irrelevance_205: author of a book asked; only a video-game score calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 471 OUT264

✓

NoToolForChessTournamentWinnerBFCL/irrelevance_195: who won a tournament asked; only a board-position move-suggester is available (the game-history fetch_game_stats lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 468 OUT620

✓

NoToolForCreateJigsawBFCL/irrelevance_204: CREATE a puzzle asked; only a puzzle-SOLVING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 461 OUT423

✓

NoToolForHaloPlayerRankBFCL/irrelevance_203: a player's RANK asked; only a player-SCORE lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 468 OUT1 424

×

NoToolForLakersTopScorerBFCL/irrelevance_193: top scorer of a team asked; only a team-details tool (roster/results/upcoming, not scorers) is available (sports_team.get_top_scorer lives in E3) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 954 OUT2 545

✓

NoToolForMonopolyWinnerBFCL/irrelevance_197: who won a casual game asked; only a possible-moves calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 466 OUT511

✓

NoToolForTopTennisPlayerBFCL/irrelevance_190: top ranked PLAYER asked; only a TEAM standing/ranking tool is available (the athlete-ranking get_top_ranked lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 465 OUT913

✓

NoToolForWorldSeriesWinnerBFCL/irrelevance_202: who won a real game asked; only a score calculator (from raw points) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 463 OUT1 374

Tool Calling - Irrelevance: Sports Statistics 9 passed

100.0%

IN31 560 OUT9 984

✓

NoToolForBasketballSeasonMvpBFCL/irrelevance_187: who won the MVP asked; only a team-ranking tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 509 OUT554

✓

NoToolForClippersRosterBFCL/irrelevance_179: roster of a team asked; only a two-team game-results tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 502 OUT976

✓

NoToolForCommonChordsInPopBFCL/irrelevance_174: most common chords in a pop song asked; only a key-signature primary-triads tool is available (find_common_chords lives in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 507 OUT1 760

✓

NoToolForCricketMatchParticipantsBFCL/irrelevance_180: who is playing in cricket matches today asked; only a cricket-schedule tool (dates/matches, not participants) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 506 OUT1 467

✓

NoToolForLaLigaPlayersBFCL/irrelevance_181: who played in a whole league asked; only a two-named-team last-match-result tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 502 OUT958

✓

NoToolForMoonlightSonataComposerBFCL/irrelevance_175: composer of Moonlight Sonata asked; only a blues-scale generator is available here (all composer-lookup tools are isolated in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 505 OUT206

✓

NoToolForNbaMvpWinnerBFCL/irrelevance_184: who won an MVP award asked (no player named); only a per-player stats tool that requires a player name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 511 OUT577

✓

NoToolForTopAssistPlayerBFCL/irrelevance_178: top ASSIST player asked; only a top-GOAL-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 506 OUT3 123

✓

NoToolForWimbledonChampionBFCL/irrelevance_183: winner of a tournament asked (no person named); only a celebrity-info tool that requires a name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 512 OUT363

Tool Calling - Irrelevance: Time, Weather & Misc 2 passed1 failed1 skipped

66.7%

IN10 341 OUT4 264

×

NoToolForCityDistanceKmBFCL/irrelevance_239: DISTANCE (km) between two cities asked; only a get_date tool that returns the TIME difference between two locations is available (its 'unit' arg mentions distance, but it computes time) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN5 187 OUT2 524

–

NoToolForCurrentTimeSydneyBFCL/irrelevance_228: current time in Sydney asked; a local-time-by-city tool is available. REVIEW FLAG (fairness): get_local_time(location='Sydney') directly answers this, so a competent human would legitimately call it. BFCL nonetheless files it under irrelevance; kept as such to preserve the source label, but this item is effectively RELEVANT and should be reviewed. Marked Hard because the "no call" expectation is contradicted by the tool itself.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_local_time' directly answers 'current time in Sydney' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForTomatoCaloriesBFCL/irrelevance_222: CALORIES in a tomato asked; only a grocery-item-details lookup available. REVIEW FLAG (fairness): item_details at 'detailed' level could arguably surface nutrition info, and it requires a store_location the user never gave; BFCL labels it irrelevant (it is a grocery-store item lookup, not a nutrition-facts service). Kept as such; marked Medium as borderline.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 578 OUT1 207

✓

NoToolForWriteABookBFCL/irrelevance_217: "can you write a book" asked; only a recipe-CREATION tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN2 576 OUT533

Tool Calling - Irrelevance: Weather, Environment & Services 9 passed1 failed

90.0%

IN40 303 OUT11 317

✓

NoToolForBestCo2ReductionBFCL/irrelevance_108: the best way to reduce CO2 asked; only a numeric emission-reduction estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 660 OUT1 183

✓

NoToolForBestMonthToVisitBFCL/irrelevance_101: best month to visit Hawaii asked; only an average-monthly-temperature tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 658 OUT1 003

×

NoToolForCactusNutrientsBFCL/irrelevance_109: a plant's nutrient needs asked; only a weekly watering-needs calculator is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN7 351 OUT3 410

✓

NoToolForCurrentTimeWeatherBFCL/irrelevance_103: the current time asked; only a current-weather forecast tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 659 OUT936

✓

NoToolForGardenSpeciesCompositionBFCL/irrelevance_107: garden species composition asked; only a soil sand/silt/clay composition analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 662 OUT388

✓

NoToolForGmoYogaMatBFCL/irrelevance_111: a GMO yoga mat asked; only an eco-friendly product locator is available (GMO is not eco-friendly) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 661 OUT854

✓

NoToolForImpactfulCasesBFCL/irrelevance_94: a firm's impactful cases asked; only a single-case detail lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 669 OUT448

✓

NoToolForPaintingEiffelCaseResultBFCL/irrelevance_96: how long to paint the Eiffel Tower asked; only a law-case result lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 661 OUT688

✓

NoToolForPollinationFlowersBFCL/irrelevance_105: top pollination flower species asked; only a route elevation-profile plotter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 663 OUT1 281

✓

NoToolForTomatoFertilizerBFCL/irrelevance_106: best fertilizer for tomatoes asked; only a soil-type analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 659 OUT1 126

Tool Calling - Parallel Calls: Art, Music & Local Services 8 passed2 failed1 skipped

80.0%

IN82 380 OUT16 756

–

CalculateDrivingDistancesShortestRouteBFCL/parallel_38: two calc_distance calls — New York→Washington DC and Los Angeles→San Francisco. Tightened per methodology: the prompt explicitly enables "shortest route", so shortest_route == true is required (schema default is false).0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Skipped: Ambiguous scope (item flaw, not model behavior): the trailing 'with optional parameter shortest route enabled' can reasonably apply to only the second leg (its nearest clause) or to both; the BFCL ground truth requires it on both, but a competent human would not answer deterministically. Kept for traceability.

Skipped

IN0 OUT0

✓

CheckInstrumentAvailabilityInTwoCitiesBFCL/parallel_42: two check_instrument_availability calls — Yamaha P125 in Berlin and Madrid.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 486 OUT874

✓

FindMusicStoresInTwoCitiesBFCL/parallel_41: two find_music_instrument_store calls — San Francisco and Chicago; instruments {Yamaha acoustic guitar, Kawai piano} in any order and either casing.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 511 OUT889

✓

FindRockAndJazzConcertsInTwoCitiesBFCL/parallel_43: four concert_finder calls — {San Francisco, New York} x {rock, jazz} over the next month. time_period may be omitted (defaults to 30 days ≈ "next month"). Isolated from concert_find_nearby (parallel_44) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 806 OUT1 481

✓

GetExhibitionDetailsForTwoMuseumsBFCL/parallel_40: two museum_exhibition_detail calls — 'Wonder of Nature' at the Louvre and 'Age of Reptiles' at the British Museum. visitor_type {child, adult} (unordered). Tightened per methodology: the prompt explicitly asks for cost "for children and adult", so both visitor types are required (schema default is ['adult'] only).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 548 OUT3 165

✓

GetMostPlayedPopAndRockSongsBFCL/parallel_45: two musicCharts_getMostPlayed calls — most played Pop and Rock song in Australia. Optional duration omitted (ground truth accepts the schema default of 0).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 470 OUT2 066

✓

GetMuseumInfoForTwoCitiesBFCL/parallel_39: two museum_info_get_info calls — Washington D.C. and Paris; details {Opening hours, Adult tickets, Child tickets} in any order.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 500 OUT1 978

✓

GetRgbAndHexForThreeColorsBFCL/parallel_37: three color_converter_get_color_info calls — navy, purple, maroon; each with {RGB, HEX} (unordered).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 622 OUT1 965

×

GetTeamRankingsInTwoLeaguesBFCL/parallel_47: two get_team_ranking calls — Barcelona in UEFA Champions League and Manchester United in La Liga.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: match0 == 1 Values: match0 = 0

Failed

IN7 471 OUT1 046

×

MixPaintColorsForTwoRequestsBFCL/parallel_36: two paint_color_mixture calls — watercolor magenta and acrylic navy.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN14 200 OUT1 689

✓

WinningPercentageForTwoTeamsTwoSeasonsBFCL/parallel_46: four calculate_winning_percentage calls — {Lakers, Bulls} x {2018, 2020}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 766 OUT1 603

Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 1) 11 passed2 failed

84.6%

IN121 977 OUT32 789

✓

AverageFourNumberSetsBFCL/parallel_132: four calculate_average calls — one per list of numbers (element order preserved).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 764 OUT1 173

×

CalculateBmiFourPeopleMetricBFCL/parallel_134: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt states "All measurements are in the metric system", so unit == "metric" is required (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: match0 == 1 Values: match0 = 0

Failed

IN8 754 OUT1 570

×

CalculateCityDistancesInKilometersBFCL/parallel_135: three geo_distance_calculate calls — NY->LA->Miami->NY, in kilometers. Tightened: the tool defaults to miles but the prompt asks "in kilometers", so units == "kilometers" is REQUIRED (omission would return miles, contradicting the request — the ground truth's "omit" option is unsafe here).1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN18 795 OUT7 927

✓

CalculateGpsDistancesAlongRouteBFCL/parallel_133: three calculate_distance calls over GPS coordinate pairs (Paris->Rome->Athens-> Giza), each in kilometers. Uses the coordinate-based tool (coord1/coord2), not the city-name distance tools that are also visible.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 790 OUT3 697

✓

CompareAverageTemperatureNyAndLaBFCL/parallel_128: two average_temperature calls — New York 7 days in Fahrenheit, Los Angeles 7 days in Celsius. Tightened: the prompt explicitly requests Fahrenheit for NY, so temp_unit == "Fahrenheit" is required (also the default); Celsius is non-default and required for LA.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 358 OUT4 155

✓

CreateTwoHistogramsWithFiveBinsBFCL/parallel_129: two create_histogram calls — one per data set, 5 bins each (data order preserved).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 822 OUT1 223

✓

EstimateElephantPopulationThreeScenariosBFCL/parallel_125: three elephant_population_estimate calls — current 500 at 2%, 1.5%, 2.5% over 10y.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 699 OUT1 313

✓

FindItalianRestaurantsInNyAndLaBFCL/parallel_130: two find_restaurants calls — New York and Los Angeles, Italian, 4 results each, dietary_requirements [vegan, gluten-free]. Distinct from restaurant_find_nearby (this tool carries food_type + number, which the prompt supplies, so it is the correct choice here).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 422 OUT1 182

✓

FindRestaurantsByDietInThreeCitiesBFCL/parallel_127: three restaurant_find_nearby calls — New York, Los Angeles, Chicago, each with dietary_preference [Vegan, Gluten-free, Dairy-free]. List order follows the prompt (matches ground truth).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 545 OUT2 199

✓

PredictEvolutionDarwinAndLamarckBFCL/parallel_126: two prediction_evolution calls — African Elephant over 5000y, Darwin then Lamarck. Tightened per methodology: the prompt explicitly says "using the Darwin model", so model == "Darwin" is required for the first call (ground truth also accepts omission via the 'Darwin' default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 381 OUT1 777

✓

ProjectPopulationGrowthJapanAndIndiaBFCL/parallel_124: four population_projections calls — Japan/10y and India/20y, each at the current (default) growth rate and at an explicit rate. "current growth rate" maps to the default (growth_rate omitted, accepts null or 0.01); the explicit 1.5%/2.1% rates are required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 706 OUT2 755

✓

RouteThreeLegsAvoidingTollsBFCL/parallel_131: three map_routing_fastest_route calls — SF->Palo Alto->San Jose->home, all avoiding tolls. Tightened: the prompt explicitly says "avoiding toll roads", so avoid_tolls == true is required for every leg (non-default value stated in the prompt).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 551 OUT1 428

✓

ShortestCityDistanceBusWithAndWithoutTransferBFCL/parallel_136: two city_distance_find_shortest calls — NY->LA by bus, first without transfer then allowing transfer. Tightened: the prompt explicitly says "using a bus", so transportation == "bus" is required for both (also the default). allow_transfer: false/omitted for the first, true for the second.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 390 OUT2 390

Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 2) 13 passed1 failed

92.9%

IN126 112 OUT20 863

✓

CalculateAreaOfThreeTrianglesBFCL/parallel_143: three calc_area_triangle calls — (10,5), (15,7), (20,10).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 958 OUT1 569

✓

CalculateBinomialProbabilityBasketballBFCL/parallel_161: three calculate_binomial_probability calls — basketball shooter at 60%: (10,7), (15,10), (20,15). probability_of_success == 0.6 is required (prompt states 60%). ISOLATED here (away from prob_dist.binomial in ToolCalling-Parallel-D3): both tools compute binomial probability, and this prompt does not name a function, so having both visible would make the choice genuinely ambiguous. Kept apart so exactly one correct tool is available.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 037 OUT2 674

✓

CalculateBmiInMetersForThreePeopleBFCL/parallel_138: three calculate_BMI calls (weight kg, height meters) — John, Sarah, Mike.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 019 OUT1 204

✓

CalculateCelestialDistancesInMilesBFCL/parallel_149: two calculate_distance calls between celestial bodies (Mars-Venus, Mars-Jupiter), in miles. Uses the body-name distance tool (body1/body2). Tightened: prompt asks "in miles", so unit == "miles" is required (tool default is kilometers, so omission would be wrong).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 764 OUT1 245

✓

CalculateClockAnglesAtThreeTimesBFCL/parallel_145: three calculate_clock_angle calls — 3:15, 8:20, 11:50. Tightened: the prompt asks for results "rounded to 2 decimal places", so round_to == 2 is required (also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 939 OUT2 499

✓

CalculateFactorialOfFourNumbersBFCL/parallel_144: four math_factorial calls — 5, 3, 4, 2.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 968 OUT890

✓

CalculateLightTravelTimeThreeLegsBFCL/parallel_147: three light_travel_time calls — 4.22, 6.1, 5.88 light years. Tightened: the prompt explicitly states the speed of light (299792458 m/s), so speed_of_light == 299792458 is required (also default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 025 OUT1 708

✓

CalculateSpeedForCarBicycleTrainBFCL/parallel_148: three calculate_speed calls — car 500m/25s in km/h, bicycle 1000m/200s in m/s, train 10000m/600s in km/h. Each output unit is explicitly named in the prompt, so to_unit is required.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 040 OUT1 197

✓

FetchAllDataFieldsForTwoEmployeesBFCL/parallel_139: two employee_fetch_data calls — Tech Solutions, employees 12345 and 67890, each fetching all four data fields (order follows the prompt / ground truth).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 872 OUT517

×

FindDiCaprioDramaAndComedyMoviesBFCL/parallel_140: two imdb_find_movies_by_actor calls — Leonardo DiCaprio, Drama in 2010 and Comedy in 2012. Tightened: the prompt explicitly names each category, so category is required for both.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN9 082 OUT2 894

✓

GetImaxAndTwoDTheaterReleasesBFCL/parallel_141: two get_theater_movie_releases calls — New York IMAX over 7 days, Los Angeles 2D over 14 days. Tightened: the prompt explicitly names each format, so format is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 789 OUT1 097

✓

PlotTwoSineWavesBFCL/parallel_146: two plot_sine_wave calls with all five parameters supplied by the prompt.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 895 OUT577

✓

SortFourListsAscendingAndDescendingBFCL/parallel_137: eight array_sort calls — four lists, each sorted ascending AND descending (list element order preserved as given).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 844 OUT2 310

✓

UpdateTwoCustomersNameAndEmailBFCL/parallel_142: two update_user_info calls — users 12345 and 67890, both set name "John" and email "example@.com". update_info is a nested object. Tightened: the prompt names the 'CustomerInfo' database, so database == "CustomerInfo" is required (also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 880 OUT482

Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 3) 10 passed1 failed

90.9%

IN87 936 OUT16 255

✓

AreaUnderCurveForTwoPolynomialsBFCL/parallel_150: two mathematics_calculate_area_under_curve calls — polynomial [3,-2,1] over [-1,2] and [1,0,-1] over [0,3] (coefficient/limit order preserved).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 831 OUT2 039

✓

BinomialProbabilitiesThreeGamesBFCL/parallel_160: three prob_dist_binomial calls — (20,5), (50,15), (100,30), each with p == 0.3 (prompt states a 0.3 win probability). The prompt explicitly names 'prob_dist.binomial'; the colliding calculate_binomial_probability tool is kept in a separate script (ToolCalling-Parallel-D2).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 001 OUT2 803

✓

CalculateBmiSystemFourPeopleBFCL/parallel_154: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt says "use the metric system for all calculations", so system == "metric" is required (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 128 OUT911

✓

DiceRollProbabilitiesThreeScenariosBFCL/parallel_159: three probability_dice_roll calls — roll 4 three times and roll 2 twice on a six-sided die (die_sides default 6, accepts omission), then roll 7 twice on an eight-sided die (die_sides == 8 required, non-default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 958 OUT2 007

✓

GenerateNormalVariatesTwoConfigsBFCL/parallel_158: four random_normalvariate calls — two at (mu 5, sigma 2) and two at (mu 10, sigma 3). Duplicate calls: each configuration must appear exactly twice (multiplicity 2).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 064 OUT1 161

✓

PredictPricesForFourHousesBFCL/parallel_157: four predict_house_price calls — New York (3/2/1500), Los Angeles (4/3/2000), Chicago (2/1/1200), Miami (3/2/1800).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 252 OUT2 608

✓

RaiseTwoNumbersToPowersBFCL/parallel_152: two math_power calls — 3^5 and 2^3, no modulus (mod omitted / null).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 684 OUT732

✓

RunLinearRegressionWithAndWithoutStandardizationBFCL/parallel_155: two run_linear_regression calls — same predictors [Age, Income, Education] and target "Spending Score", first without standardization then with. standardize is explicitly requested (false then true), so it is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 876 OUT805

✓

TrainRandomForestFourConfigsBFCL/parallel_156: four random_forest_train calls on "data_random_forest" — (100,10), (200,20), (300,30), (400,40). The prompt explicitly names this function, disambiguating it from train_random_forest_classifier visible in this script.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 310 OUT1 307

✓

TrainRfClassifierTwoConfigsBFCL/parallel_153: two train_random_forest_classifier calls — dataset1 (depth 10, 100 trees) and dataset2 (depth 20, 200 trees). The prompt explicitly names this function, disambiguating it from the other random-forest training tool (random_forest_train) visible in this script.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 874 OUT556

×

TriangleAreasInDifferentUnitsBFCL/parallel_151: three geometry_area_triangle calls in different units — (15,20) square meters, (25,30) square feet, (35,40) square inches. First unit is the default (accepts omission / square meters); feet and inches are non-default and required.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: match0 == 1 Values: match0 = 0

Failed

IN7 958 OUT1 326

Tool Calling - Parallel Calls: Finance, Stats & Legal 13 passed

100.0%

IN114 989 OUT24 332

✓

BuildTwoDecisionTreeClassifiersBFCL/parallel_12: two model_DecisionTreeClassifier calls — gini/depth 5 and entropy/depth 10, both random_state 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 440 OUT634

✓

CalculateFutureValueForThreeTermsBFCL/parallel_18: three financials_calculate_future_value calls — years 1, 5, 10 (same 1000 @ 0.05).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 602 OUT1 945

✓

CalculatePresentValueForThreeTermsBFCL/parallel_14: three calculate_present_value calls — years 10, 20, 30 (same payment 1000, rate 0.05).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 599 OUT1 988

✓

CapitalGainsTaxForTwoStatesBFCL/parallel_15: two calculate_capital_gains_tax calls — CA (15000/25000) and FL (20000/50000).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 492 OUT2 441

✓

ComputePythagorasForTwoTrianglesBFCL/parallel_10: two math_pythagoras calls — sides (3,4) and (5,12).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 383 OUT1 383

✓

ConfidenceIntervalForTwoSampleSizesBFCL/parallel_13: two confidence_interval_calculate calls — sample sizes 50 and 150. Tightened per methodology: the prompt explicitly requests "95% confidence interval", so confidence_level == 0.95 is required (schema default is 0.9, but the prompt states the value).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 495 OUT2 989

✓

CourtCaseStatusAndTrialDatesBFCL/parallel_22: four court_info_get_case_status calls — cases XY1234 (LA) and GH5678 (Orange County), each queried for status (details may be omitted via the 'status' default) and for trial_date.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 028 OUT3 345

✓

GetStockPriceAndVolumeForThreeCompaniesBFCL/parallel_17: three get_stock_data calls — AAPL, GOOG/GOOGL, MSFT; each with data_points {price, volume} (unordered).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 534 OUT1 133

✓

LoanEligibilityForTwoInstitutionsBFCL/parallel_20: two loan_eligibility_check calls — HSBC (500k/100k) and Wells Fargo (700k/120k).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 507 OUT1 067

✓

MortgagePaymentForThreeLoanTermsBFCL/parallel_19: three calculate_mortgage_payment calls — loan terms 15, 20, 30 (400000 @ 0.04).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 627 OUT3 760

✓

PredictHousePriceInTwoCitiesBFCL/parallel_11: two ml_predict_house_price calls — New York 3000 sq ft, Los Angeles 4000 sq ft.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 414 OUT1 425

✓

ReturnOnInvestmentGainAndLossBFCL/parallel_16: two calculate_return_on_investment calls — gain 500 on 2000, loss -1000 on 5000.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 445 OUT933

✓

SearchMoneyLaunderingConvictionsBFCL/parallel_21: two law_crimes_search calls — money laundering in San Francisco 2019 and Texas 2018.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 423 OUT1 289

Tool Calling - Parallel Calls: Legal, History & Environment 11 passed2 failed1 skipped

84.6%

IN127 322 OUT27 555

✓

BookTwoConnectingFlightsBFCL/parallel_30: two book_flight calls — San Francisco→Tokyo (2022-05-03) and Tokyo→Sydney (2022-05-18).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 057 OUT615

✓

CalculateAlimonyForTwoDurationsBFCL/parallel_23: two alimony_calculator_ca_calculate calls — durations 10 and 20 years (10000/3000 income).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 120 OUT4 485

×

CalculateWasteForFamilyAndBachelorBFCL/parallel_29: two waste_calculation_calculate calls — LA family (2 adults, 2 children) and New York bachelor (1 single). The population argument is a nested object.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: match1 == 1 Values: match1 = 0

Failed

IN9 070 OUT3 176

✓

FetchTwoHistoricalFactsBFCL/parallel_31: two history_fact_fetch calls — Treaty of Paris and Magna Carta. Optional depth/year omitted (ground truth accepts the schema defaults).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 976 OUT1 092

✓

FindClassicalConcertsWithParkingBFCL/parallel_44: two concert_find_nearby calls — classical concerts near Berlin and Paris next Friday. Tightened per methodology: the prompt explicitly asks for concerts "with available parking", so amenities == ['Parking'] is required (this also matches the schema default, but the prompt states it). Isolated from concert_finder (parallel_43) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 034 OUT2 177

–

FindCourtCasesByNumberAndTypeBFCL/parallel_26: four court_case_find calls — the cross product of case numbers {67813, 71249} and types {Civil, Criminal}; case_number is a single-element array; Civil may be omitted via its default. FLAGGED as borderline (see report): the prompt "case numbers '67813','71249' for type 'Civil' and 'Criminal'" is ambiguous about the pairing; BFCL ground truth expects all four combinations, kept faithful.0 AI queries 0 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Skipped: Ambiguous call decomposition (item flaw, not model behavior): 'case numbers 67813, 71249 for type Civil and Criminal' has no single deterministic reading - two calls (both numbers per type, case_number as a list) is as valid as the BFCL 2x2 = four single-number calls; a competent human would not answer deterministically. Kept for traceability.

Skipped

IN0 OUT0

✓

FindNatureReservesNearTwoCitiesBFCL/parallel_27: two nature_reserve_find_nearby calls — Berkeley (10km, picnic tables + restrooms) and Tokyo (5km, playgrounds + biking trails). Amenity list order is unordered.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 058 OUT736

✓

GetCaseDetailsWithHistoryAndLitigantsBFCL/parallel_24: two law_case_get_details calls — cases 28473 and 64725. Tightened per methodology: the prompt explicitly asks for "their history and details of litigants", so include_history == true and include_litigants == true are required (schema defaults are false).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 040 OUT1 421

✓

GetEventsForTwoPresidenciesBFCL/parallel_32: two us_history_events_by_presidency calls — Abraham Lincoln and George Washington. Optional start_year/end_year omitted (ground truth accepts the schema defaults).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 971 OUT1 329

✓

GetPresidentsAndVicePresidentsByYearBFCL/parallel_33: four get_president_and_vp calls — president for 1980 and 2016, vice president for 1975 and 2011.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 326 OUT2 116

✓

GetReligionInfoForTwoEmpiresBFCL/parallel_35: two ancient_empires_get_religion_info calls — Persian Empire and Mauryan Empire. Tightened per methodology: the prompt explicitly asks for "religious history and influences", so include_influences == true is required (schema default is false).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 011 OUT2 428

✓

GetTemperatureForTwoCitiesBFCL/parallel_28: two get_current_and_future_temperature calls — Seattle and Los Angeles, next 3 hours.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 983 OUT1 010

×

LookupLawsuitsForTwoYearsBFCL/parallel_25: two lawsuit_lookup calls — Dara Inc for years 2019 and 2018.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN18 622 OUT3 210

✓

TrackChristianityInTwoRegionsBFCL/parallel_34: two religion_history_track calls — Christianity in Egypt and Turkey, 100–1500 A.D.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 054 OUT3 760

Tool Calling - Parallel Calls: Markets & Environment 12 passed1 failed

92.3%

IN114 500 OUT30 654

✓

CalculateCagrForTwoInvestmentsBFCL/parallel_176: two calculate_cagr calls — (5000,7000,5) and (8000,12000,3).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 665 OUT1 888

✓

CalculateEmissionSavingsForSolarAndWindBFCL/parallel_186: two calculate_emission_savings calls — solar(12 months) and wind(8 months), Midwest region. Prompt states the Midwest region explicitly -> region required (accept "Midwest"/"Midwest region").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 558 OUT3 153

✓

CompareFutureValueWithAndWithoutContributionsBFCL/parallel_182: two finance_calculate_future_value calls — (5000,0.07,10, no contribution) and (3000,0.06,10, contribution 200). First scenario states "no additional contributions" -> null or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 725 OUT2 630

×

ComputeEuclideanNormForThreePointsBFCL/parallel_183: three math_hypot calls — (5,7), (10,15), (20,25). z omitted or 0 (2D points).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 2 expectedCount = 3

Failed

IN8 620 OUT2 714

✓

EstimatePopulationsForTwoSpeciesAcrossTwoYearsBFCL/parallel_185: four estimate_population calls — Bengal Tigers/India and African Elephants/Kenya for 2021 and for 2023. NOTE: upstream ground truth's third entry lists year "" (omitted -> default 2020), which is an upstream error: the prompt explicitly asks for 2023 for "both these species". We tighten both 2023 estimates to year == 2023 (fair deterministic answer).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 904 OUT2 169

✓

FindRootsForTwoQuadraticEquationsBFCL/parallel_184: two algebra_quadratic_roots calls — (3,7,2) and (5,-4,1).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 678 OUT1 430

✓

GetClosingPricesForTwoCompaniesOnTwoDatesBFCL/parallel_178: four get_stock_price (by-date variant) calls — the 2x2 cross product of {Microsoft, Apple} x {2022-01-01, 2022-02-01} on NASDAQ. Upstream ground truth's first entry loosely lists company_name as [Microsoft, Apple]; the deterministic human answer is the four distinct (company, date) combinations, which we require. exchange NASDAQ (default) may be omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 862 OUT4 161

✓

GetDetailedAirQualityForThreeDaysBFCL/parallel_187: three get_air_quality calls — New York City, detail true, for 2023-05-05, 05-04, 05-03. Prompt asks for PM2.5/PM10/ozone/sources -> detail == true; "today/yesterday/day before" with today = May 5, 2023.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 764 OUT1 431

✓

GetFourPriceTypesForTwoCompaniesBFCL/parallel_180: eight stock_price calls — {Microsoft, Apple} x {Open, Close, High, Low}, days 30. Prompt explicitly lists all four data types, so data_type must be set explicitly (required) to distinguish the eight calls; omission (default 'Close') cannot represent the four requested types.1 AI query 8 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Passed

IN9 438 OUT1 798

✓

GetPricePerOunceForFourMetalsBFCL/parallel_177: four get_metal_price calls — gold, silver, platinum, palladium, measure 'ounce'. Prompt asks "price per ounce" -> measure == "ounce" required.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 785 OUT2 085

✓

GetStockPricesForFourCompaniesAndDurationsBFCL/parallel_181: four get_stock_prices calls — Apple(1 week), Microsoft(2 weeks), Amazon(3 weeks), Tesla(1 month). Each 'companies' argument is a single-element list.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 834 OUT1 347

✓

GetTrafficInfoForThreeLegsWithModesBFCL/parallel_188: three get_traffic_info calls — NY->LA (driving), LA->SF (bicycling), SF->NY (transit). First leg explicitly says "driving" -> mode == "driving" required (tightened; also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 759 OUT1 983

✓

ProjectFutureValueForThreeStocksBFCL/parallel_175: three portfolio_future_value calls — AAPL(5000,0.07,5), MSFT(8000,0.06,7), AMZN(10000,0.08,10).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 908 OUT3 865

Tool Calling - Parallel Calls: Mathematics & Physics (set 1) 13 passed

100.0%

IN104 045 OUT20 125

✓

AreaOfTwoRightTrianglesInMetersBFCL/parallel_68: two calculate_area calls — right triangles base/height 12x15 and 18x24 in meters. unit required (tool default is 'cm'; the prompt states meters/square meters).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 988 OUT1 025

✓

CheckPlayerStatusAcrossThreeSeasonsBFCL/parallel_49: three player_status_check calls — player 3142 in RocketLeague for seasons 2017/2018/2019. Seasons are explicitly given, so each is required (default 2023 not applicable).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 071 OUT940

✓

CircumferenceOfFourCirclesInCmBFCL/parallel_65: four calculate_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt gives every radius in cm (e.g. "5cm") and the tool default unit is 'm', so a correct call must pass unit == 'cm'/'centimeter' (ground truth requires it on the first call and merely allows omission on the rest; we require it consistently since it is stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 213 OUT2 043

✓

ConvertEuroAndPoundsToDollarsBFCL/parallel_59: two get_conversion_cost calls — 15000 Euro->dollars and 200 pounds->dollars.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 900 OUT2 559

✓

FinalVelocityForFourDropsBFCL/parallel_81: four calculate_final_velocity calls — heights 10/20/15/25, initial_velocity 0 (stated), gravity 9.81 (default, may be omitted).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 398 OUT2 718

✓

FindItalianAndJapaneseRestaurantsBFCL/parallel_52: two restaurant_finder calls — Italian in New York (preferences ['Vegetarian']) and Japanese in Los Angeles (delivery; ['Delivery'] or omitted since default is ['Delivery']).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 896 OUT1 536

✓

FindRootsOfTwoQuadraticsAlgebraBFCL/parallel_62: two algebra_quadratic_roots calls — 3x^2+4x+2 and 5x^2-7x+3.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 050 OUT2 279

✓

FindSoupAndVeganSaladRecipesBFCL/parallel_51: two recipe_search_find calls — Chicken Noodle Soup (diet omitted/default 'Keto') and a Vegan Salad (diet == 'Vegan', dish name accepts several phrasings).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 831 OUT860

✓

GcdOfJohnAndMaryNumbersBFCL/parallel_80: two math_gcd calls — gcd(36,48) and gcd(60,96). Same tool as parallel_76.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 946 OUT1 465

✓

GcdOfTwoPairsBFCL/parallel_76: two math_gcd calls — gcd(45,60) and gcd(81,27).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 878 OUT1 063

✓

GetPokemonMovesAndCheckSolarBeamBFCL/parallel_48: two PokemonGO_get_moves calls — Pikachu's move list (move omitted/default 'Run') and a check whether Bulbasaur can learn 'Solar Beam'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 868 OUT1 236

✓

PrimeFactorsFormattedAndArrayBFCL/parallel_74: four get_prime_factors calls — numbers 4567 and 7890, each formatted true and false (user wants "a formatted string as well as an array").1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 160 OUT2 026

✓

SaveGameProgressEasyAndHardBFCL/parallel_50: two game_save_progress calls — stage 7 easy and stage 3 hard; level omitted/default 'user'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 846 OUT375

Tool Calling - Parallel Calls: Mathematics & Physics (set 2) 13 passed

100.0%

IN107 921 OUT24 667

✓

AreaOfThreeCirclesInMetersBFCL/parallel_66: three geometry_area_circle calls — radii 5/10/15 in meters (default is meters, so units may be omitted).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 335 OUT1 257

✓

BookTwoHotelRoomsBFCL/parallel_57: two hotel_booking_book calls — Sheraton (NY, 2 adults/1 child) and Marriott (LA, 1 adult/2 children).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 383 OUT880

✓

CheckAvailabilityForTwoHotelsBFCL/parallel_56: two hotel_booking_check_availability calls — Queens Hotel (Berlin) and Royal Hotel (Paris), each with check-in/out dates (several date formats accepted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 315 OUT1 119

✓

CircumferenceOfFourCirclesInMetersBFCL/parallel_70: four geometry_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt asks for output in meters and gives radii in m, but the tool default unit is 'cm', so a correct call must pass units == 'm'/'meters' (ground truth requires it on the first call and only allows omission on the rest; we require it consistently since it is explicitly stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 515 OUT2 387

✓

DerivativeOfPolynomialAndItsDerivativeBFCL/parallel_71: two calculate_derivative calls — 3x^3-2x^2+5x-7 at x=4, then its derivative 9x^2-4x+5 at x=2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 230 OUT3 714

✓

FinalVelocityForThreeDragRaceRoundsBFCL/parallel_83: three final_velocity calls — (v0=0,a=5,t=10), (v0=10,a=7,t=8), (v0=20,a=4,t=12).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 512 OUT1 346

✓

FindGroceryStoresByCriteriaBFCL/parallel_55: two grocery_store_find_by_criteria calls — NYC ['24 hours'] and San Diego ['Home Delivery'].1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 148 OUT1 569

✓

GetTwoCookingRecipesBFCL/parallel_53: two get_cooking_recipe calls — Lasagne Bolognese for 4 and Caesar Salad for 2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 166 OUT2 728

✓

GetTwoExchangeRatesBFCL/parallel_58: two get_exchange_rate calls — USD->AUD and USD->CAD.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 130 OUT1 039

✓

HighestCommonFactorOfFourPairsBFCL/parallel_77: four math_hcf calls — (45,60), (90,120), (36,48), (72,96). FLAGGED (see final report): the prompt is phrased as chaining ("use that result to find the HCF with...") but the ground truth treats it as four INDEPENDENT parallel calls on the literal pairs; a human might interpret it as sequential composition. Kept BFCL-faithful (four independent literal-pair calls) since parallel semantics preclude feeding one call's result into another.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 549 OUT2 843

✓

OrderSameItemsFromTwoWholeFoodsLocationsBFCL/parallel_54: two whole_foods_order calls — same two items (order-independent) at downtown and uptown, size large.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 196 OUT725

✓

PrimeFactorizeAsDictionaryBFCL/parallel_79: two prime_factorize calls — 4567 and 7890, return_type 'dictionary' (explicitly requested).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 172 OUT3 628

✓

SolveTwoQuadraticEquationsBFCL/parallel_63: two solve_quadratic_equation calls — (5,6,1) and (3,2,1). Prompt names this function explicitly.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 270 OUT1 432

Tool Calling - Parallel Calls: Mathematics & Physics (set 3) 12 passed

100.0%

IN98 280 OUT17 972

✓

AreaOfTwoCirclesInMetersBFCL/parallel_67: two geometry_calculate_area_circle calls — radii 5 and 10 in meters. unit required (tool default is 'cm'; the prompt states meters).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 019 OUT1 134

✓

AreaOfTwoTrianglesBFCL/parallel_69: two calculate_triangle_area calls — base/height 10x5 and 8x6.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 079 OUT1 167

✓

DerivativeOfTwoFunctionsAtValuesBFCL/parallel_73: two calculus_derivative calls — 3x^2+2x-1 at 5 (variable x, default) and 4y^3-3y^2+2y-1 at 3 (variable y).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 124 OUT1 171

✓

DisplacementForThreeCarsBFCL/parallel_84: three calculate_displacement calls — (15,7,accel 3.5), (20,10,accel 2.0), (25,8,no acceleration). The third car travels "without any acceleration" so acceleration == 0 (also the default; omission accepted).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 349 OUT3 003

✓

EuclideanNormForThreePointsBFCL/parallel_61: three math_hypot calls — (3,4) and (6,8) with z omitted/0, and (9,12,15).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 254 OUT1 267

✓

FactorialOfThreeNumbersBFCL/parallel_60: three math_factorial calls — 5, 7, 9.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 081 OUT1 006

✓

FinalSpeedForTwoFreeFallObjectsBFCL/parallel_85: two calculate_final_speed calls — initial_speed 0 for 10 s and initial_speed 5 for 7 s; gravity -9.81 (default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 147 OUT2 262

✓

GcdViaNumberTheoryBFCL/parallel_78: two number_theory_gcd calls — (45,60) and (81,63). Prompt names this function explicitly.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 022 OUT745

✓

IntegrateTwoFunctionsWithTwoMethodsBFCL/parallel_72: four integrate calls — x**3 on [2,5] and 2x**2+3x-1 on [-1,3], each with trapezoid (default, may be omitted) and simpson (explicit).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 468 OUT2 063

✓

PrimeFactorsOfThreeNumbersBFCL/parallel_75: three number_analysis_prime_factors calls — 45, 100, 150.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 087 OUT993

✓

SolveFourQuadraticsWithRootTypeBFCL/parallel_64: four solve_quadratic calls — (2,5,3) all roots, (1,-3,2) real, (4,-7,3) all roots, (1,2,1) real. 'all' is the default so it may be omitted; 'real' is explicitly requested and required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 538 OUT2 183

✓

VelocityForTwoCyclingDaysBFCL/parallel_82: two calculate_velocity calls — 120 km in 5 h and 150 km in 6 h; unit km/h (default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 112 OUT978

Tool Calling - Parallel Calls: Mixed Domains 9 passed1 failed

90.0%

IN73 363 OUT16 314

✓

CalculateBmiForTwoPeopleBFCL/parallel_4: two calculate_bmi calls — 6 ft / 80 kg and 5.6 ft / 60 kg.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 278 OUT2 057

✓

CalculateEmForceForTwoTimeIntervalsBFCL/parallel_1: two calculate_em_force calls — same field/area, d_time 4 then 10.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 299 OUT1 237

✓

CalculateFactorialForThreeNumbersBFCL/parallel_7: three math_factorial calls — 5, 10, 15.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 338 OUT1 268

✓

CalculateResistanceForCopperAndAluminumBFCL/parallel_2: two calculate_resistance calls — copper then aluminum, same length/area. Tightened per methodology: the prompt explicitly names "resistivity of copper", so the first call must pass resistivity == "copper" (ground truth also accepts omission via the 'copper' default, but an explicitly-stated value must be supplied). Aluminum is non-default → required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 286 OUT1 455

✓

CalculateSalesTaxForThreeCitiesBFCL/parallel_6: three calculate_sales_tax calls — Chicago/IL $30.45, Sacramento/CA $52.33, Portland/OR $11.23.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 470 OUT1 420

✓

FetchPopulationForCitiesStateAndCountryBFCL/parallel_8: four database_us_census_get_population calls — NYC (city), LA (city), Alaska (state), USA (country); year defaults to 2000 (may be omitted). Requires area-type inference.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 554 OUT1 725

×

FindMovieShowingsForTwoMoviesBFCL/parallel_9: two find_movie_showing calls near San Diego — Tenet at 5 pm and No Time To Die at 7:30 pm.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 1 expectedCount = 2

Failed

IN7 158 OUT2 101

✓

GetSequenceAnd3DForThreeProteinsBFCL/parallel_3: three protein_info_get_sequence_and_3D calls — HbA1c, normal + rat hemoglobin. Tightened per methodology: the prompt explicitly asks for "their 3D models", so model_3d == true is required (ground truth also accepts omission via the default of true).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 417 OUT2 628

✓

ListShowRatingsOnNetflixAndHuluBFCL/parallel_5: two streaming_services calls — Netflix['Friends'] and Hulu['The Office','Stranger Things'] (Hulu show order unordered), both sorted by rating. Must group shows per service.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 301 OUT2 034

✓

PlayTwoArtistsForGivenDurationsBFCL/parallel_0: two spotify_play calls — Taylor Swift for 20 min and Maroon 5 for 15 min.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 262 OUT389

Tool Calling - Parallel Calls: Physical & Life Sciences (set 1) 12 passed

100.0%

IN102 016 OUT26 387

✓

ElectricFieldForTwoChargesBFCL/parallel_94: two calculate_electric_field calls — charge 5/distance 2 and charge 3/distance 4. "Conducted in a vacuum" = the default permitivity → may be omitted or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 410 OUT1 786

✓

ElectromagneticForceInTwoMediaBFCL/parallel_96: two electromagnetic_force calls — same charges 5,10 and distance 2, differing by medium_permittivity. Vacuum call (8.854e-12) equals the default → may be omitted or 8.854e-12; the second medium (5e-12) is non-default → required.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 566 OUT2 712

✓

EstimateTravelTimeForThreeRoutesBFCL/parallel_93: three route_estimate_time calls — (1) NYC->Miami via Philadelphia/Washington D.C./Atlanta, (2) NYC->Miami skipping Atlanta (via Philadelphia/Washington D.C.), (3) Philadelphia-> Miami via Washington D.C. Stop orderings from ground truth accepted. FLAG parallel_93: BORDERLINE. Ground truth allows many stop-list variants, some including the start city itself as a stop; a competent human's stop set is not fully deterministic. Kept BFCL-faithful by accepting the natural stop orderings (start city excluded from stops).1 AI query 3 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Passed

IN8 561 OUT3 795

✓

FinalSpeedForTwoDroppedObjectsBFCL/parallel_88: two calculate_final_speed calls — v0 0/height 10 and v0 5/height 20. gravity stated as 9.8 (the default) so it may be omitted or 9.8 per ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 416 OUT2 283

✓

FinalVelocityForTwoCarsBFCL/parallel_87: two calculate_final_velocity calls — Car A (v0 0, a 6, t 10), Car B (v0 20, a 4, t 15).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 433 OUT1 305

✓

FinalVelocityFromDistanceForTwoObjectsBFCL/parallel_86: two kinematics_final_velocity_from_distance calls — accel 5/dist 100 and accel 10/dist 200. "Both objects start from rest" → initial_velocity may be omitted or 0 (default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 405 OUT1 941

✓

FindVeganRestaurantsInThreeCitiesBFCL/parallel_91: three vegan_restaurant_find_nearby calls — LA, SF, Seattle, each with operating_hours 22 (prompt: "open until at least 22:00", a non-default value → required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 462 OUT1 505

✓

GenerateFourTravelItinerariesBFCL/parallel_90: four travel_itinerary_generator calls — Tokyo/7/200/urban, Paris/10/150/history, Sydney/5/100/nature, Rome/12/180/culture. Tokyo's "urban" equals the default → omit or "urban".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 787 OUT2 862

✓

GetDirectionsForRoundTripBFCL/parallel_89: three get_directions calls — SF->Palo Alto (fastest), Palo Alto->Golden Gate Bridge (scenic), Golden Gate Bridge->SF (fastest). route_type explicitly stated each time.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 512 OUT1 836

✓

MagneticFieldForTwoLoopsBFCL/parallel_95: two calculate_magnetic_field calls — current 10/radius 0.5 and current 15/radius 1.0. "Same as in free space" = default permeability → may be omitted or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 514 OUT1 618

✓

ResonantFrequencyForTwoCircuitsBFCL/parallel_97: two calculate_resonant_frequency calls — L 0.005/C 1e-07/round 3 and L 0.007/C 2e-07/round 4. Rounding is stated explicitly → required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 480 OUT2 725

✓

ShortestDrivingDistanceRoundTripBFCL/parallel_92: three get_shortest_driving_distance calls — NYC->LA, LA->Miami, Miami->NYC, all in miles (prompt explicitly "in miles" → non-default unit required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 470 OUT2 019

Tool Calling - Parallel Calls: Physical & Life Sciences (set 2) 12 passed1 failed

92.3%

IN112 061 OUT28 026

✓

AbsolutePressureAtTwoAltitudesBFCL/parallel_102: two calc_absolute_pressure calls — gauge 2.5 with atm 1 (default; may be omitted) and gauge 2.5 with atm 0.85 (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 519 OUT3 570

×

BoilingMeltingPointsAcrossSeaLevelsBFCL/parallel_100: four get_boiling_melting_points calls — water@0, iron@1000, water@1000, iron@0. FLAG parallel_100: BORDERLINE. The prompt ("water and iron at sea levels of 0 and 1000 meters respectively") reads most naturally as TWO calls (water@0, iron@1000), but the BFCL ground truth is the full 2x2 cross product (FOUR calls). Kept BFCL-faithful to the ground truth; a competent human could reasonably answer with two calls, so this item's expected call count is not fully deterministic.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN8 700 OUT1 923

✓

CellInfoDetailedAndBriefBFCL/parallel_108: two biology_get_cell_info calls — neuron with detailed=true (prompt: "detailed description"), muscle with detailed=false ("less detailed", the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 408 OUT2 801

✓

DensityForTwoSubstancesBFCL/parallel_101: two calculate_density calls (mass/volume schema) — 10 kg / 2 m³ and 15 kg / 3 m³. unit "kg/m³" equals the default → may be omitted or "kg/m³".1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 536 OUT1 158

✓

ElectricFieldStrengthAtSeveralDistancesBFCL/parallel_98: four calculate_electric_field_strength calls — charge 2, distances 0.5/1/2 in vacuum (default medium), plus distance 1 in air. The distance-1 vacuum and distance-1 air calls are distinguished by medium. Vacuum equals the default → may be omitted or "vacuum"; "air" is required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 837 OUT4 024

✓

EntropyChangeForTwoConditionsBFCL/parallel_103: two entropy_change_calculate calls — substance A, mass 2, temps 25->75 and 10->50. pressure 1 equals the default → may be omitted or 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 642 OUT1 744

✓

FetchDnaSequenceThreeWaysBFCL/parallel_106: three fetch_DNA_sequence calls for id XYZ123 — default fasta/0, then genbank format, then 500 bp upstream. Defaults (fasta, upstream 0) may be omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 613 OUT892

✓

FunctionLookupForTwoMoleculesBFCL/parallel_110: two cell_biology_function_lookup calls — ATP in mitochondria and DNA in nucleus, both with specific_function=true (prompt asks for a specific function within the organelle).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 469 OUT1 072

✓

HeatCapacityForThreeConditionsBFCL/parallel_105: three calc_heat_capacity calls — 300K/2.5m³, 350K/2.5m³, 300K/1.5m³, all for air. "air" equals the default gas → may be omitted or "air".1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 717 OUT1 573

✓

MolecularWeightForTwoCompoundsBFCL/parallel_111: two calculate_molecular_weight calls — Glucose (C6H12O6) and Sucrose (C12H22O11), both in grams/mole (prompt explicitly requests that unit; ground truth also accepts "g/mol").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 470 OUT1 339

✓

PhaseChangeEnergyForWaterBFCL/parallel_99: four thermo_calculate_energy calls — 500 g water, phase melting/freezing/ vaporization/condensation. Substance "water" equals the default → may be omitted or "water".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 879 OUT2 959

✓

ProteinSequencesForTwoGenesTwoSpeciesBFCL/parallel_107: four get_protein_sequence calls — BRCA1 and BRCA2 in Homo sapiens (default species, may be omitted) and in Pan troglodytes (non-default → required).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 708 OUT3 474

✓

ProteinsInThreeCompartmentsBFCL/parallel_109: three cellbio_get_proteins calls — nucleus/mitochondria/cytoplasm, each with include_description=true (prompt asks for "a brief description of each protein").1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 563 OUT1 497

Tool Calling - Parallel Calls: Physical & Life Sciences (set 3) 13 passed

100.0%

IN113 522 OUT24 543

✓

AnalyzeDnaForThreeMutationTypesBFCL/parallel_114: six analyze_dna_sequence calls — two sequences ("AGCTTAGCTA" and "AGCTTAGGCTA") against reference "AGCTTAGCTA", each checked for insertion (default, may be omitted), deletion, and substitution mutations.1 AI query 6 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Passed

IN9 201 OUT1 848

✓

DiabetesPredictionForFourPeopleBFCL/parallel_113: four diabetes_prediction calls for four people with distinct weight/height/activity.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 962 OUT1 673

✓

EntropyChangeIsothermalAndNotBFCL/parallel_104: two calculate_entropy_change calls — same 300->350 K and heat_capacity 4.18, once isothermal (the default true, may be omitted) and once not isothermal (false, required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 624 OUT3 593

✓

ForestGrowthForTwoRegionsBFCL/parallel_120: two forest_growth_forecast calls — Amazon Rainforest over 10 years and Boreal Forests of Canada over 20 years, excluding human impact (false = the default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 515 OUT2 429

✓

GenerateFourDnaSequencesBFCL/parallel_123: four generate_DNA_sequence calls — length 500 with a single-nucleotide preference of A, T, C, G respectively.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 822 OUT1 422

✓

GeneticSimilarityInTwoFormatsBFCL/parallel_115: four genetics_calculate_similarity calls — human vs chimpanzee and human vs gorilla, each in percentage (default, may be omitted) and fraction. species1 accepts "human"/"Human".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 794 OUT1 892

✓

GenotypeFrequenciesForThreeGenotypesBFCL/parallel_116: three calculate_genotype_frequency calls — allele frequency 0.7, genotypes AA/Aa/aa.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 679 OUT1 635

✓

IdentifyFourBirdsBFCL/parallel_119: four identify_bird calls — small blue/forest (default size, may be omitted), large black/lake, medium brown/desert, large green/tropical rainforest.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 918 OUT1 927

✓

MutationTypeForTwoSnpsBFCL/parallel_112: two mutation_type_find calls — rs123456 in Homo sapiens (default species, may be omitted) and rs7891011 in Canis lupus familiaris (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 537 OUT2 643

✓

PopulationDensityForChinaTwoYearsBFCL/parallel_117: two calculate_density calls (country/population schema) — China in 2000 (pop 1.267 billion) and 2010 (pop 1.341 billion), land area 9.597 million km² both times.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 608 OUT1 090

✓

PrecipitationStatsForThreeTimeFramesBFCL/parallel_118: three ecology_data_precipitation_stats calls — Amazon rainforest for six_months, year, five_years.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 610 OUT1 321

✓

TurtlePopulationForTwoYearsBFCL/parallel_121: two ecology_get_turtle_population calls — Galapagos Islands in 2015 and 2020, both with species info requested (species=true; years stated explicitly → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 530 OUT1 639

✓

VehicleEmissionsForThreeVehiclesBFCL/parallel_122: three calculate_vehicle_emission calls — 15000 miles each: gas with default factor 1.4 (may be omitted), diesel with factor 2.7, EV with factor 0.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 722 OUT1 431

Tool Calling - Parallel Calls: Services & Lookups 12 passed

100.0%

IN98 416 OUT20 469

✓

CalculateShortestAndScenicDistancesForTwoLegsBFCL/parallel_190: four calculate_shortest_distance calls — {NYC->LA, LA->Miami} x {Shortest, Scenic}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 368 OUT2 333

✓

FetchNewsForTwoTopicsInEuropeBFCL/parallel_192: two get_news calls — Climate Change and Artificial Intelligence, 5 articles each, region Europe. Region "Europe" explicitly requested -> required (accept "Europe"/"EU").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 049 OUT1 642

✓

FindLibrariesWithFacilitiesInThreeCitiesBFCL/parallel_191: three public_library_find_nearby calls — NY[Reading Room,Fiction], LA[Wi-Fi,Children Section], Chicago[Cafe,Reading Room]. Facilities compared as an unordered pair.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 267 OUT2 550

✓

FindParksWithAmenitiesInThreeCitiesBFCL/parallel_189: three parks_find_nearby calls — NY[Tennis Court,Picnic Area], LA[Playground,Running Track], Chicago[Tennis Court,Playground]. Amenities compared as an unordered pair.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 248 OUT1 360

✓

FindUpcomingEventsInThreeCitiesBFCL/parallel_194: three event_finder_find_upcoming calls — jazz/LA(14d), rock/Chicago(10d), classical/Boston(7d). All day windows explicitly stated -> days_ahead required (7 for Boston, which also equals the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 261 OUT1 095

✓

GetCurrentWeatherForFourCitiesBFCL/parallel_199: four get_current_weather calls — New York, Los Angeles, London, Tokyo, with both temperature and humidity included. Prompt explicitly asks to include both -> flags == true (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 343 OUT1 986

✓

GetDetailedBriefsForTwoMoviesBFCL/parallel_195: two movie_details_brief calls — Inception and The Dark Knight, extra_info true. Prompt asks for Director/Cast/Awards -> extra_info == true required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 027 OUT1 797

✓

GetLawsuitDetailsForTwoCasesBFCL/parallel_196: two get_lawsuit_details calls — case 12345 (NY Supreme Court, with verdict) and case 67890 (LA Superior Court, no verdict). "would like the verdict" -> true; "do not need" -> null/false.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 176 OUT2 792

✓

GetLawsuitInfoForTwoCasesByYearBFCL/parallel_197: two lawsuit_info calls — 12345ABC (2018, New York) and 67890XYZ (2019, California). Years are explicitly stated -> required. First location "New York" may be omitted (matches the default).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 167 OUT1 427

✓

GetStockPricesForTwoCompaniesOverDaysBFCL/parallel_179: two get_stock_price (by-days variant) calls — Apple on NASDAQ for 10 days, Microsoft on NYSE for 15 days. Both exchanges are explicitly named -> required (NASDAQ / NYSE). The tool's 'company' field is a ticker symbol; company names are accepted as synonyms.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 050 OUT1 638

✓

SearchLawsuitsForTwoEntitiesInCaliforniaBFCL/parallel_198: two lawsuit_search calls — Google/Santa Clara and Facebook/San Mateo, state California. Prompt states "both in the state of California" -> state required (accept "California"/"CA").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 087 OUT1 056

✓

SendTwoEmailsWithCcAndBccBFCL/parallel_193: two send_email calls — Project Update to john.doe (cc manager, bcc hr) and Meeting Reminder to jane.doe (cc assistant, bcc hr). Bodies matched exactly.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 373 OUT793

Tool Calling - Parallel Calls: Statistics & Finance 12 passed1 failed

92.3%

IN125 260 OUT33 259

✓

CalculateBinomialProbabilitiesForBasketballShotsBFCL/parallel_163: three calc_binomial_prob calls — (10,6,0.6), (10,6,0.5), (15,6,0.5).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 157 OUT1 784

×

CalculateCompoundInterestForThreePeriodsBFCL/parallel_170: three calculate_compound_interest calls — principal 5000, rate 0.025, time 2/3/5. Prompt states "compounded quarterly, 4 times in a year" -> n == 4 required (tightened; also ground truth).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN16 731 OUT4 609

✓

CalculateDiscountedCashFlowForTwoBondsBFCL/parallel_169: two calculate_discounted_cash_flow calls — (50,5,0.05) and (60,7,0.04). Prompt explicitly states "face value of the bond is $1000" for both -> face_value == 1000 required (ground truth also accepts omission via the default, but an explicitly-stated value must be supplied).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 994 OUT2 138

✓

CalculateEventProbabilitiesForCardDeckBFCL/parallel_162: three probability_of_event calls — Ace 4/52 as decimal, heart 13/52 as decimal, red card 26/52 as ratio. format_as_ratio: "as a decimal" -> false or omitted; "as a ratio" -> true (required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 124 OUT1 854

✓

CalculateQuarterlyDividendForThreeScenariosBFCL/parallel_168: three finance_calculate_quarterly_dividend_per_share calls — (5M,2M), (6M,2M), (6M,2.5M).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 180 OUT1 862

✓

CalculateReturnOnEquityForTwoCompaniesBFCL/parallel_171: two calculate_return_on_equity calls — (1M,5M,200k) and (2M,10M, no dividends). Second company "did not pay any dividends" -> dividends_paid null or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 955 OUT1 718

✓

CalculateRoiForThreeStocksBFCL/parallel_174: three calculate_return_on_investment calls — (150,180,20), (200,210,30), (250,300,40). Dividends are explicitly stated per stock -> required.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 096 OUT1 799

✓

ComputeRSquaredForTwoRegressionModelsBFCL/parallel_167: two linear_regression_get_r_squared calls on the same CSV — swap education/credit_score as dependent variable. Independent-variable order follows the prompt/ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 950 OUT1 370

✓

PredictFutureValueForTwoInvestmentsBFCL/parallel_172: two finance_predict_future_value calls — (5000,0.05,10) and (7000,0.04,15). Prompt states "compound interest annually" -> compounding_periods_per_year == 1 required (tightened; also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 964 OUT1 623

✓

PredictHousePricesForThreeHousesBFCL/parallel_166: three predict_house_price calls — NY 2500/3/2000, LA 3000/3/2005, Chicago 2000/2/1995. The second house is described as "similar" with only area/year/location changed, so rooms == 3 per ground truth.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 147 OUT4 779

✓

PredictProfitForTwoFundsBFCL/parallel_173: two investment_predictProfit calls — (5000,0.07,5) and (8000,0.05,7).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 916 OUT3 039

✓

RunChiSquaredTestForTwoContingencyTablesBFCL/parallel_164: two chi_squared_test calls — tables [45,55,35,65] and [30,70,50,50], alpha 0.05. Prompt states "significance level of 0.05 for both" -> alpha == 0.05 required (tightened). Table order is fixed.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 900 OUT2 628

✓

RunTTestAtTwoSignificanceLevelsBFCL/parallel_165: two t_test calls — same datasets, alpha 0.05 then 0.01 (both stated -> required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 146 OUT4 056

Tool Calling - Simple: Arts, Venues & Music (set 1) 7 passed1 failed

87.5%

IN50 785 OUT11 994

✓

SimpleConcertDetailsBeyonceBFCL/simple_python_286 (concert.get_details): artist=Beyonce, location=San Diego, date=April 2022.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 358 OUT1 264

×

SimpleConcertInfoPopNyBFCL/simple_python_284 (concert_info.get): location=New York, date=next month, genre=Pop.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.date?.ToLower()).AnyOf(["next month", "2023-06-01", "06/01/2023", "jun.1,2023", "june 2023"])

Failed

IN6 333 OUT3 093

✓

SimpleGenerateMelodyC4BFCL/simple_python_291 (music_generator.generate_melody): key=C, start_note=C4, length=16, tempo=120 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 381 OUT1 220

✓

SimpleKeySignatureCsharpBFCL/simple_python_298 (music_theory.key_signature): key='C#', scale_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 318 OUT776

✓

SimpleMuseumHoursLouvreBFCL/simple_python_273 (museum.get_hours): name='Louvre Museum', location=Paris, day omitted or Monday.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 325 OUT1 761

✓

SimplePaintNeededWall30x12BFCL/simple_python_259 (calculate_paint_needed): coverage_rate=400, length=30, height=12.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 381 OUT889

✓

SimpleReligionHistoryChristianity16cBFCL/simple_python_252 (get_religion_history): religion=Christianity, century=16, sort_by=importance, count=10.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 358 OUT1 257

✓

SimpleSculptureInfoPlensaBFCL/simple_python_263 (get_sculpture_info): artist_name='James Plensa', detail=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 331 OUT1 734

Tool Calling - Simple: Arts, Venues & Music (set 2) 7 passed1 failed

87.5%

IN57 369 OUT19 414

✓

SimpleComposeMelodyCFGBFCL/simple_python_292 (compose_melody): progression=[C,F,G], measures=4, instrument=Piano or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 321 OUT519

✓

SimpleConcertFindDetailsWeekndBFCL/simple_python_290 (concert.find_details): artist=The Weeknd, month=December, year=2022 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 297 OUT5 990

✓

SimpleFindConcertRockChicagoBFCL/simple_python_285 (find_concert): location=Chicago, price=100, genre=Rock.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 307 OUT1 191

✓

SimpleMuseumInfoMetBFCL/simple_python_274 (museum_info): museum_name=Metropolitan Museum of Art, info_type=opening_hours or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 295 OUT1 649

✓

SimpleMusicalScaleCsharpBFCL/simple_python_299 (musical_scale): key='C#'/'C sharp', scale_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 286 OUT1 835

✓

SimplePaintRequirementWallExclusionBFCL/simple_python_260 (paint_requirement.calculate): area {width=20,height=12}, paint_coverage=350, exclusion {type=window, area=15}.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN6 416 OUT2 433

✓

SimpleReligionHistoryChristianity300To400BFCL/simple_python_254 (get_religion_history): religion=Christianity, start_year=300, end_year=400, event_type=all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 322 OUT2 086

×

SimpleSculptureDetailsDavidBFCL/simple_python_264 (sculpture.get_details): artist=Michelangelo, title=David, detail=size.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 125 OUT3 711

Tool Calling - Simple: Arts, Venues & Music (set 3) 9 passed

100.0%

IN60 208 OUT15 679

✓

SimpleChordProgressionKeyBFCL/simple_python_297 (music.theory.chordProgression): progression=[I,V,vi,IV], bools any.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 693 OUT1 056

✓

SimpleCompoundInterest10kBFCL/simple_python_269 (calculate_compound_interest): principle=10000, interest_rate=0.05, time=10, compounds_per_year=1 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 736 OUT2 021

✓

SimpleConcertSearchClassicalLaBFCL/simple_python_287 (concert.search): genre=classical, location=Los Angeles/LA, date='this weekend', price_range=cheap.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 692 OUT2 579

✓

SimpleCreateMixCMajorBFCL/simple_python_293 (music_composer.create_mix): scale='C Major', note_duration=quarter, track_length=180.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 715 OUT1 128

✓

SimpleGenerateCircleImageRedBFCL/simple_python_256 (generate_circle_image): radius=50, color=Red, background=white or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 684 OUT565

✓

SimpleMuseumWorkingHoursLouvreBFCL/simple_python_276 (museum_working_hours.get): museum=Louvre, location=Paris, day=Monday or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 670 OUT2 081

✓

SimpleRetrieveReligionInfoBuddhismBFCL/simple_python_253 (retrieve_religion_info): religion_name=Buddhism, detail_level=full.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 665 OUT2 077

✓

SimpleSculptureSearchChicago19cBFCL/simple_python_265 (sculpture_search): location=Chicago, time_frame='19th century', material=all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 682 OUT2 293

✓

SimpleTimeDifferenceSfSydneyBFCL/simple_python_250 (get_time_difference): place1=San Francisco/SF, place2=Sydney.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 671 OUT1 879

Tool Calling - Simple: Arts, Venues & Music (set 4) 9 passed

100.0%

IN56 909 OUT17 282

✓

SimpleBuildingDimensionsEmpireStateBFCL/simple_python_270 (building.get_dimensions): building_name=Empire State Building, unit=feet.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 320 OUT814

✓

SimpleConcertFindNearbySeattleBFCL/simple_python_289 (concert.find_nearby): location=Seattle, genre=jazz.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 309 OUT1 548

✓

SimpleCreateChordProgressionCBFCL/simple_python_294 (music_generation.create_chord_progression): key=C, chords=4, progression_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 324 OUT1 760

✓

SimpleEarliestReferenceJesusBFCL/simple_python_251 (get_earliest_reference): name='Jesus Christ', source='historical records'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 318 OUT1 649

✓

SimpleFindInstrumentAcousticBFCL/simple_python_280 (find_instrument): budget=1000, type=acoustic, make omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 317 OUT876

✓

SimpleGuitarPriceLesPaulBFCL/simple_python_283 (guitar_price.find): model='Gibson Les Paul', condition=Excellent, location=Chicago.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 331 OUT2 812

✓

SimpleIdentifyColorSeaGreenBFCL/simple_python_257 (identify_color_rgb): color_name='Sea Green', standard=basic or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 307 OUT1 730

✓

SimpleMuseumInfoBritishDateBFCL/simple_python_277 (museum_info): museum='The British Museum', date=2023-06-20, information=[opening_hours,ticket_price] (either order).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN6 358 OUT3 165

✓

SimpleSculptureValueThinkerBFCL/simple_python_266 (get_sculpture_value): sculpture='The Thinker', artist=Rodin.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 325 OUT2 928

Tool Calling - Simple: Arts, Venues & Music (set 5) 9 passed

100.0%

IN57 649 OUT12 204

✓

SimpleCircleDimensionsRadius5BFCL/simple_python_272 (calculate_circle_dimensions): radius=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 384 OUT1 296

✓

SimpleDrawRectangleRedBFCL/simple_python_261 (draw_rectangle): width=20, height=10, color=red.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 412 OUT260

✓

SimpleFindFluteYamahaBFCL/simple_python_282 (find_flute): brand=Yamaha, specs=[open hole, C foot, silver headjoint].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 416 OUT1 564

✓

SimpleGenerateScaleProgressionCBFCL/simple_python_296 (music_generator.generate_scale_progression): key=C, tempo=80, duration=4, scale_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 419 OUT956

✓

SimpleInstrumentDetailsPianoYamahaBFCL/simple_python_278 (get_instrument_details): instrument=piano, manufacturer=Yamaha, features=[price,rating] (either order).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 399 OUT2 481

✓

SimpleMetTopArtworksBFCL/simple_python_275 (metropolitan_museum.get_top_artworks): number=5, sort_by=popularity or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 410 OUT1 433

✓

SimpleMixPaintYellowBlueBFCL/simple_python_258 (mix_paint_color): color1=yellow, color2=blue, lightness=60.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 406 OUT731

✓

SimplePapalBiographyInnocentIIIBFCL/simple_python_255 (religious_history.get_papal_biography): papal_name=Innocent III, include_contributions=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 395 OUT1 764

✓

SimpleSculptureLocatorMichelangeloBFCL/simple_python_268 (sculpture_locator.find_by_artist): artist=Michelangelo, material=Marble, location=Rome.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 408 OUT1 719

Tool Calling - Simple: Arts, Venues & Music (set 6) 6 passed1 failed

85.7%

IN41 876 OUT7 454

✓

SimpleAnalyzeStructureB1004BFCL/simple_python_271 (analyze_structure): building_id=B1004, floors=[2,3,4], mode=dynamic.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 006 OUT730

✓

SimpleConcertBookingEminemBFCL/simple_python_288 (concert_booking.book_ticket): artist=Eminem, city=New York City, num_tickets=2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 966 OUT321

✓

SimpleFindExhibitionNyBFCL/simple_python_267 (find_exhibition): location='New York City, NY', art_form=sculpture, month omitted, user_ratings=high.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 973 OUT1 264

✓

SimpleInstrumentInfoViolin1721BFCL/simple_python_281 (get_instrument_info): name=Violin, maker=Stradivarius, year=1721.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 000 OUT1 179

×

SimpleInstrumentPriceStratocasterBFCL/simple_python_279 (instrument_price.get): brand=Fender, model='American Professional II Stratocaster', finish=Rosewood.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.finish?.ToLower()).AnyOf(["rosewood"])

Failed

IN5 981 OUT2 010

✓

SimpleModifyPaintingOilBFCL/simple_python_262 (modify_painting): size='12x18', medium=oil, dominant_color=red.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 983 OUT667

✓

SimpleSongLyricsBohemianBFCL/simple_python_295 (get_song_lyrics): song_title='Bohemian Rhapsody', artist_name=Queen, lang=English or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 967 OUT1 283

Tool Calling - Simple: Food, Travel & Shopping (set 1) 7 passed1 failed

87.5%

IN51 764 OUT10 168

✓

SimpleCookingConvertPoundsToOuncesBFCL/simple_python_365 (cooking_conversion.convert): 2 pounds->ounces of butter.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 466 OUT925

✓

SimpleCurrencyExchangeUsdToCadBFCL/simple_python_388 (currency_exchange.convert): USD->CAD, amount=500.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 466 OUT1 327

×

SimpleGroceryStoreOrganicHoustonBFCL/simple_python_369 (grocery_store.find_nearby): Houston, categories {Organic,Fruits,Vegetables} any order.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.categories.Count >= 2 Values: args.categories.Count = 1

Failed

IN6 459 OUT880

✓

SimpleHotelBookPetFriendlyManhattanBFCL/simple_python_380 (hotel_booking): Manhattan, single, 3 nights, 2023-03-10, pet_friendly.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN6 529 OUT1 254

✓

SimpleMapsDistanceEiffelLouvreBFCL/simple_python_394 (maps.get_distance_duration): Eiffel Tower -> Louvre Museum, traffic omitted/false.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 467 OUT1 682

✓

SimpleRecipeFindByDietMealBFCL/simple_python_353 (find_recipes): diet=gluten-free, meal_type=dinner, ingredients omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 447 OUT1 637

✓

SimpleRestaurantFinderItalianNYCBFCL/simple_python_361 (restaurant_finder): city=NYC, cuisine=Italian, diet=Gluten-free.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 462 OUT823

✓

SimpleTimeZoneLondon24hBFCL/simple_python_376 (time_zone_converter): city=London, country=UK, display_format=24h.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 468 OUT1 640

Tool Calling - Simple: Food, Travel & Shopping (set 2) 7 passed2 failed

77.8%

IN70 881 OUT13 762

✓

SimpleBestSushiPlacesTokyoBFCL/simple_python_362 (get_best_sushi_places): city=Tokyo, top=5, review_rate=4.0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 054 OUT1 584

✓

SimpleCurrencyConverterUsdToGbpBFCL/simple_python_389 (currency_converter): USD->GBP, amount=200.0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 051 OUT1 321

✓

SimpleCurrentTimeSydneyCityBFCL/simple_python_377 (get_current_time): city=Sydney, country=Australia, format omitted/HH:MM:SS.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 020 OUT1 216

✓

SimpleDistanceCalcNyToBostonTerrainBFCL/simple_python_397 (distance_calculator.calculate): New York -> Boston, consider_terrain=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 036 OUT696

✓

SimpleHiltonAvailabilityParisBFCL/simple_python_381 (hilton_hotel.check_availability): Paris, 2023-04-04..08, 2 adults, chain Hilton/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 111 OUT1 007

×

SimpleRecipeCaloriesBeefLasagnaBFCL/simple_python_355 (recipe_info.get_calories): website=Foodnetwork.com, recipe=Beef Lasagna, meal_time omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 739 OUT3 708

✓

SimpleRecipeUnitTbspToTspBFCL/simple_python_366 (recipe.unit_conversion): 2 tablespoon->teaspoon, precision omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 041 OUT942

×

SimpleSafewayOrderPaloAltoBFCL/simple_python_370 (safeway.order): Palo Alto, [olive oil, rice], [3,1]. EXPECTED MODEL MISS (kept, do not loosen): "a five pound bag of rice" = quantity 1 (5 is the weight, not the count); models tend to send quantity 5. This is a genuine model error the test correctly catches.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 796 OUT1 239

✓

SimpleVeganSoupRecipeUnder30BFCL/simple_python_354 (get_vegan_recipe): dish_type=soup, cooking_time=30, ingredient_preference omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 033 OUT2 049

Tool Calling - Simple: Food, Travel & Shopping (set 3) 8 passed

100.0%

IN51 968 OUT11 177

✓

SimpleBookHotelHiltonChicagoBFCL/simple_python_382 (book_hotel): Hilton Hotel, Chicago, single, 2022-12-10, 2 nights.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN6 543 OUT504

✓

SimpleCookingTimeRoastChickenBFCL/simple_python_368 (calculate_cooking_time): weight_kg=1.5, method omitted/roast, temp omitted/180.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 496 OUT1 361

✓

SimpleCurrencyConvertEurToCadBFCL/simple_python_390 (currency_conversion.convert): 150 EUR->CAD.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 483 OUT1 564

✓

SimpleCurrentTimeSydneyLocationBFCL/simple_python_379 (get_current_time): location=Sydney, country=Australia, timezone omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 472 OUT1 813

✓

SimpleParkingLotCentralParkBFCL/simple_python_395 (parking_lot.find_nearest): Central Park NY, radius=2, type omitted/public.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 488 OUT1 389

✓

SimpleRecipeFinderVeganServes2BFCL/simple_python_356 (recipe_finder.find): servings=2, diet=vegan, prep_time=30.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 508 OUT1 151

✓

SimpleRestaurantClosestSushiBostonBFCL/simple_python_363 (restaurant_search.find_closest): Boston, Sushi, amenities=[Patio].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 489 OUT1 968

✓

SimpleWholeFoodsPriceLABFCL/simple_python_371 (whole_foods.check_price): Los Angeles, [tomatoes, lettuce].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 489 OUT1 427

Tool Calling - Simple: Food, Travel & Shopping (set 4) 7 passed

100.0%

IN41 988 OUT8 920

✓

SimpleBookRoomThePlazaBFCL/simple_python_383 (book_room): The Plaza, single, 2 nights.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 000 OUT448

✓

SimpleConvertCurrencyJpyToUsdBFCL/simple_python_393 (convert_currency): 20000 JPY->USD.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 004 OUT2 238

✓

SimpleFindRestaurantItalianBrooklynBFCL/simple_python_364 (find_restaurant): Brooklyn, Italian, Gluten-free.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 995 OUT910

✓

SimpleGetRecipeVeganChocCakeBFCL/simple_python_357 (get_recipe): dish_name=(vegan) chocolate cake, diet_preference=vegan.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 987 OUT1 582

✓

SimpleHospitalLocateDenverBFCL/simple_python_396 (hospital.locate): Denver CO, radius=5, department=Pediatrics.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 002 OUT1 285

✓

SimpleTimezoneConvertNyToLondonBFCL/simple_python_378 (timezone.convert): 3pm, New York -> London.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 998 OUT1 118

✓

SimpleWholeFoodsTopBrandsBananasBFCL/simple_python_372 (whole_foods.find_top_brands): product=bananas, number omitted/5, organic=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 002 OUT1 339

Tool Calling - Simple: Food, Travel & Shopping (set 5) 5 passed1 failed

83.3%

IN35 450 OUT8 153

✓

SimpleExchangeRateFeeGbpJpyBFCL/simple_python_391 (get_exchange_rate_with_fee): GBP->JPY, fee=0.02.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 894 OUT1 215

✓

SimpleHotelBookingBookParisBFCL/simple_python_384 (hotel_booking.book): Paris, 07-10-2022..07-20-2022, 2 adults, 1 child, Standard/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 959 OUT1 068

✓

SimpleMuseumHoursMetSaturdayBFCL/simple_python_398 (get_museum_hours): museum=Metropolitan Museum of Art/The Met, day=Saturday.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 873 OUT1 852

×

SimpleRecipeSearchGfCookieBFCL/simple_python_358 (recipe_search): diet=[gluten free], time_limit=30, dish=cookie.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.dish?.ToLower()).AnyOf(["cookie"])

Failed

IN5 886 OUT1 252

✓

SimpleRestaurantSearchRatedNYCBFCL/simple_python_399 (restaurant_search): NYC, Italian, rating=4, accepts_credit_cards=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 908 OUT1 044

✓

SimpleWalmartPurchaseSanJoseBFCL/simple_python_373 (walmart.purchase): San Jose, [apples, rice, (bottled) water], pack_size=[1,1,12].1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 930 OUT1 722

Tool Calling - Simple: Food, Travel & Shopping (set 6) 5 passed

100.0%

IN27 810 OUT5 696

✓

SimpleGameHighestScoringPlayerBFCL/simple_python_350 (get_highest_scoring_player): game=Valorant, season=2022.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 554 OUT867

✓

SimpleHotelBookRoomKingLABFCL/simple_python_385 (hotel_bookings.book_room): LA, king size, 15-10-2023, 2 nights, rooms omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 599 OUT543

✓

SimpleLatestExchangeGbpJpyBFCL/simple_python_392 (latest_exchange_rate): GBP->JPY, amount omitted/1.0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 540 OUT1 691

✓

SimpleNutritionalInfoAvocadoBFCL/simple_python_374 (grocery_info.nutritional_info): Walmart, avocado, [Protein,Calories,Carbohydrates].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 561 OUT1 197

✓

SimpleRecipeSearchVegPastaBFCL/simple_python_359 (recipe_search): dietary_restriction=Vegetarian, ingredients=[pasta,cheese], servings=2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 556 OUT1 398

Tool Calling - Simple: Food, Travel & Shopping (set 7) 3 passed1 failed

75.0%

IN20 844 OUT3 849

✓

SimpleBookHotelLuxuryParadiseBFCL/simple_python_386 (book_hotel): Hotel Paradise, Las Vegas, luxury, 05-12-2022, 3 days, view=city.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 263 OUT427

✓

SimpleFindRecipeCaloriesCarbonaraBFCL/simple_python_360 (find_recipe): recipeName=pasta carbonara, maxCalories=500.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 193 OUT1 006

✓

SimpleGameMultiplayerFinderBFCL/simple_python_351 (multiplayer_game_finder): platform=Windows 10, rating=4.5, genre omitted/Action.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 196 OUT1 184

×

SimpleWalmartPriceCheckPumpkinsBFCL/simple_python_375 (walmart.check_price): [pumpkins,eggs], quantities [3,24] or [3,2], store omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.items[1].ToLower()).AnyOf(["eggs", "dozen eggs"])

Failed

IN5 192 OUT1 232

Tool Calling - Simple: Food, Travel & Shopping (set 8) 2 passed1 failed

66.7%

IN14 598 OUT5 389

×

SimpleFindRecipeVeganBrowniesBFCL/simple_python_367 (find_recipe): dietary_restrictions=vegan, recipe_type=dessert (brownies), time=30.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.recipe_type?.ToLower()).AnyOf(["dessert"])

Failed

IN4 848 OUT2 200

✓

SimpleGameAverageUserScoreBFCL/simple_python_352 (gamespot.getAverageUserScore): game=Zelda BotW, platform=Nintendo Switch/all platforms.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 858 OUT2 175

✓

SimpleHotelBookingPlazaRangeBFCL/simple_python_387 (hotel_booking): Plaza Hotel, New York NY, 2022-06-01..2022-06-04, rooms omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 892 OUT1 014

Tool Calling - Simple: Knowledge, Places & History (set 1) 10 passed

100.0%

IN69 302 OUT12 918

✓

SimpleAirQualityChicagoBFCL/simple_python_203 (get_air_quality): location=Chicago, detail=true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 923 OUT787

✓

SimpleEvolutionScientistBFCL/simple_python_242 (get_scientist_for_discovery): discovery=Theory of Evolution.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 914 OUT1 016

✓

SimpleGermanWarEventsBFCL/simple_python_231 (history.get_key_events): country=Germany/DE, 1871-1945, event_type=[War].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 959 OUT1 818

✓

SimpleKingOfFranceBFCL/simple_python_230 (historic_leader_search): location=France, date=1510, title omitted or King.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 928 OUT1 927

✓

SimplePandaPopulationBFCL/simple_python_201 (estimate_population): species=panda(s), country=China/CN, year omitted or 2024.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 915 OUT1 471

✓

SimplePersonalityTraitsTypeBFCL/simple_python_227 (get_personality_traits by type): type=ENFJ, traits=[strengths, weaknesses].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 934 OUT1 292

✓

SimplePresidentCivilWarBFCL/simple_python_238 (us_history.get_president): event=American Civil War, year in 1861-1865.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 925 OUT734

✓

SimpleRestaurantNearbyBFCL/simple_python_204 (restaurant.find_nearby): location=Seattle[, WA], cuisine=Chinese, max_distance=10.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 936 OUT1 446

✓

SimpleTordesillasDateBFCL/simple_python_233 (european_history.get_event_date): event=Treaty of Tordesillas, format=YYYY.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 939 OUT937

✓

SimpleTrafficBostonNewYorkBFCL/simple_python_205 (get_traffic_info): start=Boston, end=New York/NYC, mode omitted or driving.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 929 OUT1 490

Tool Calling - Simple: Knowledge, Places & History (set 2) 8 passed2 failed

80.0%

IN80 552 OUT13 467

✓

SimpleBitcoinNewsBFCL/simple_python_210 (get_news): topic=Bitcoin, quantity=5, region omitted or US.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 890 OUT1 480

✓

SimpleEuropeanWars19cBFCL/simple_python_234 (history_eu.fetch_events): century=19, region=any of N/S/E/W, category=Wars. EXPECTED MODEL MISS (kept, do not loosen): region is an enum (Northern/Southern/Eastern/Western); the prompt says only "European", so any of the four is accepted, but models often send the invalid value "Europe". Failing to respect the enum is a genuine model error the test correctly catches.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 905 OUT1 722

✓

SimpleLibraryNearbyBFCL/simple_python_209 (public_library.find_nearby): location=Boston MA/Massachusetts, facilities=[Fiction, Wi-Fi] (any order).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 913 OUT1 104

✓

SimpleMonarchOfEnglandBFCL/simple_python_232 (monarch.getMonarchOfYear): location=England/ENG, year=1800, fullName=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 907 OUT956

×

SimpleNeutronDiscovererBFCL/simple_python_243 (get_discoverer): discovery=neutron, detail=true.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 487 OUT1 433

×

SimpleParksNearbyBFCL/simple_python_206 (parks.find_nearby): location=London[, UK], amenities=[Tennis Court].1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 903 OUT1 691

✓

SimplePersonalityTraitsHobbyBFCL/simple_python_228 (get_personality_traits by hobby): hobby=jogging, trait_count=3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 889 OUT1 461

✓

SimplePopulationGrowthBFCL/simple_python_221 (population_growth_estimate): location=London, years=5, rate omitted or 1.2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 892 OUT1 691

✓

SimplePresidentDuringEventBFCL/simple_python_241 (US_President_During_Event): event=Civil War, country omitted or USA.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 885 OUT797

✓

SimpleTreatyLisbonDateBFCL/simple_python_235 (get_event_date): event=Treaty of Lisbon (variants), location omitted or Lisbon[, Portugal].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 881 OUT1 132

Tool Calling - Simple: Knowledge, Places & History (set 3) 9 passed1 failed

90.0%

IN72 333 OUT10 463

✓

SimpleAppleStockInfoBFCL/simple_python_212 (get_stock_info): company=Apple Inc., detail_level=detailed, market omitted or NASDAQ.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 219 OUT1 271

✓

SimpleCivilWarStartDateBFCL/simple_python_236 (us_history.get_event_info): event=American Civil War/Civil War, specific_info=Start Date.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 220 OUT659

✓

SimpleFlightBookBFCL/simple_python_213 (flight.book): departure=San Francisco/SF, destination=London, date=2022-04-27 (variants), time=afternoon, direct=true.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 274 OUT868

×

SimpleGravitationPublicationYearBFCL/simple_python_244 (publication_year.find): author=Isaac Newton, work=Law of Universal Gravitation (variants), location omitted or all.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.work_title.ToLower().AnyOf(["law of universal gravitation", "universal law of gravitation", "the law of universal gravitation"])

Failed

IN7 228 OUT1 527

✓

SimpleMovieBriefBFCL/simple_python_215 (movie_details.brief): title=Interstellar, extra_info omitted or false.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 209 OUT1 107

✓

SimplePresidentIn1861BFCL/simple_python_239 (US_president.in_year): year=1861, full_name omitted or true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 237 OUT988

✓

SimpleRadiumDiscovererBFCL/simple_python_245 (discoverer.get): element_name=radium (quote variants), year omitted or 0, first omitted or true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 199 OUT863

✓

SimpleSendEmailBFCL/simple_python_211 (send_email): to=john.doe@example.com, subject=Meeting, body (opt trailing period), cc/bcc empty.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 276 OUT808

✓

SimpleShortestDistanceBFCL/simple_python_207 (calculate_shortest_distance): start=New York (variants), end=Miami (variants), route_preference=Shortest.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 243 OUT1 649

✓

SimpleUpcomingConcertsBFCL/simple_python_214 (event_finder.find_upcoming): location=New York (variants), genre=Rock, days_ahead=30.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 228 OUT723

Tool Calling - Simple: Knowledge, Places & History (set 4) 10 passed

100.0%

IN71 799 OUT12 884

✓

SimpleBmiCalcBFCL/simple_python_222 (calculate_bmi): weight=70, height=180, unit omitted or metric.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 179 OUT1 392

✓

SimpleCarEmissionsBFCL/simple_python_200 (calculate_emissions): distance=12000, fuel_type=gas, fuel_efficiency omitted or 25.0, efficiency_reduction omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 213 OUT1 798

✓

SimpleFmriAnalyzeBFCL/simple_python_217 (fMRI.analyze): data_source path, sequence_type=multi-band, smooth=6, voxel_size=2.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 222 OUT758

✓

SimpleGetDirectionsBFCL/simple_python_208 (map_service.get_directions): start=New York/NYC, end=Los Angeles/LA, avoid=[highways, tolls] (any order).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 174 OUT992

✓

SimpleGravityDiscoveryBFCL/simple_python_246 (science_history.get_discovery_details): discovery=Gravity, method_used omitted or default.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 139 OUT1 558

✓

SimpleGroupDynamicsBFCL/simple_python_223 (group_dynamics.pattern): total=50, extroverts=15, introverts=35.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 199 OUT1 536

✓

SimpleMostFollowedBFCL/simple_python_224 (social_media_analytics.most_followed): topic=psychology, sub_topics=[behaviour, group dynamics], region omitted or all.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 170 OUT1 643

✓

SimpleNeuronCoordinatesBFCL/simple_python_219 (get_neuron_coordinates): neuron_type=GABA, brain_region=All/all part of the brain/entire brain.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 174 OUT1 581

✓

SimplePresidentIn1940BFCL/simple_python_240 (history_api.get_president_by_year): year=1940, full_term_only omitted/true/false.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 151 OUT680

✓

SimpleSentimentAnalysisBFCL/simple_python_216 (sentiment_analysis): text=the review, language=english/en.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 178 OUT946

Tool Calling - Simple: Knowledge, Places & History (set 5) 10 passed

100.0%

IN73 074 OUT11 869

✓

SimpleBigFiveScoresBFCL/simple_python_229 (get_bigfive_scores): characteristics=[efficient, organized, easy going, compassionate], scale omitted or medium.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 304 OUT1 245

✓

SimpleChristianityHistoryBFCL/simple_python_249 (religion.history_info): religion=Christianity, till_century=14, include_people omitted or false.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 293 OUT1 488

✓

SimpleEinsteinContributionBFCL/simple_python_247 (historical_contrib.get_contrib): scientist=Albert Einstein, date=1915-03-17 (variants), category omitted or all.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 310 OUT928

✓

SimpleHistoricalGdpBFCL/simple_python_237 (get_historical_GDP): country=United States/US, start_year=1960, end_year=2000.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 315 OUT812

✓

SimpleMriReportBFCL/simple_python_218 (patient.get_mri_report): patient_id=546382, mri_type omitted or brain, status=concluded.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 324 OUT986

✓

SimpleNeuronalActivityBFCL/simple_python_220 (calculate_neuronal_activity): input_synaptic_rate=200, weight=0.5, decay_rate=0.1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 334 OUT1 390

✓

SimpleReadingPreferenceBFCL/simple_python_225 (psych_research.get_preference): category=reading, option_one=digital[reading], option_two=physical book(s), demographic omitted or all.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 297 OUT1 301

✓

SimpleRelativityInventionBFCL/simple_python_248 (science_history.get_invention): invention_name=theory of relativity, want_year=true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 287 OUT698

✓

SimpleRenewableEmissionSavingsBFCL/simple_python_202 (calculate_emission_savings): energy_type=renewable, usage_duration=3, region=California/CA. EXPECTED MODEL MISS (kept, do not loosen): the prompt says "renewable energy sources" generically; models often over-specify a concrete type (e.g. "solar"). Ground truth wants "renewable".1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 315 OUT2 108

✓

SimpleZodiacCompatibilityBFCL/simple_python_226 (get_zodiac_compatibility): sign1=Aries, sign2=Gemini, scale omitted or percentage.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 295 OUT913

Tool Calling - Simple: Law, Finance & Records (set 1) 9 passed1 failed

90.0%

IN76 852 OUT17 619

✓

SimpleAirQualityByDateBFCL/simple_python_196 (air_quality): location London, date reformatted to 08-16-2022.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 287 OUT2 725

✓

SimpleCivilCaseDetailsWithDissentBFCL/simple_python_167 (law.civil.get_case_details): case_title Roe v. Wade, include_dissent true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 287 OUT1 824

✓

SimpleCompoundedInterestQuarterlyBFCL/simple_python_153 (calculate_compounded_interest): principal 5000, rate 0.03, time 5, n 4 (quarterly).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 323 OUT1 259

✓

SimpleCrimeRateByCityBFCL/simple_python_164 (get_crime_rate): city San Francisco, state California/CA, type violent/omitted, year 2020.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 297 OUT2 214

✓

SimpleCriminalRecordsByPersonBFCL/simple_python_158 (get_criminal_records): name Mr. X, location New York, NY, from_year 2012, to_year 2015.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 318 OUT1 100

✓

SimpleDetailedWeatherForecastHoursBFCL/simple_python_185 (detailed_weather_forecast): location New York, duration 72 hours, include_precipitation true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 290 OUT1 872

✓

SimpleFutureValueByDurationBFCL/simple_python_155 (calculate_future_value): initial_investment 1000, interest_rate 0.05, duration 2, compounded omitted or 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 309 OUT1 966

✓

SimpleLawsuitDetailsWithExtraFieldsBFCL/simple_python_178 (get_lawsuit_details): case_number LAX2019080202, court_location Los Angeles, additional_details omitted or full default list.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 307 OUT882

×

SimpleLawsuitSearchByFilingDateBFCL/simple_python_168 (lawsuit_search): company Google/GOOG, start_date 01-01-2021, location California, status ongoing/omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 159 OUT2 410

✓

SimpleLawyerExperienceMonthsBFCL/simple_python_175 (lawyer.get_experience): name John Doe, law_type Bankruptcy.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 275 OUT1 367

Tool Calling - Simple: Law, Finance & Records (set 2) 7 passed3 failed

70.0%

IN74 534 OUT13 305

✓

SimpleAirQualityIndexByTimeBFCL/simple_python_197 (get_air_quality_index): location San Diego, time 12pm/12:00.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 055 OUT1 904

×

SimpleCivilCasesRetrieveTheftBFCL/simple_python_165 (civil_cases.retrieve): year 2020, crime_type theft, location Los Angeles.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 785 OUT1 253

×

SimpleFindNearbyDivorceLawyerBFCL/simple_python_166 (lawyer.find_nearby): city Chicago IL, specialty [Divorce], fee 400. NOTE: single-call contract kept intentionally; the model occasionally emits a spurious second call (no other visible tool fits this query), which is a model-reliability signal, not a test defect.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.city?.ToLower()).AnyOf(["chicago, il.", "chicago, il"])

Failed

IN7 080 OUT1 309

✓

SimpleGenerateLawContractRentalBFCL/simple_python_162 (generate_law_contract): parties [John, Alice], contract_type Rental Agreement, location California/CA.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 069 OUT1 272

✓

SimpleLawsuitDetailsFindByCompanyBFCL/simple_python_176 (lawsuit_details.find): company_name Apple Inc., year 2010, case_type Patent/IPR.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 081 OUT1 511

✓

SimpleLawsuitDetailsWithVerdictBFCL/simple_python_181 (get_lawsuit_details): case_number 123456-ABC, court_location Los Angeles, with_verdict true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 093 OUT1 184

✓

SimpleLocalNurseryAnnualPlantsBFCL/simple_python_193 (local_nursery.find): location Toronto, plant_types [Annual].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 058 OUT1 459

×

SimplePropertyRecordsWithOwnerBFCL/simple_python_163 (property_records.get): address 123 main street, parcel_number 1234567890, county Santa Clara, include_owner true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.county?.ToLower()).AnyOf(["santa clara"])

Failed

IN7 142 OUT1 011

✓

SimpleRoiSavingsAccountBFCL/simple_python_150 (calculate_roi): deposit 1000, annual_interest_rate 0.03, years 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 103 OUT1 469

✓

SimpleWeatherForecastDetailedDaysBFCL/simple_python_189 (weather_forecast_detailed): location New York, days 3, details true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 068 OUT933

Tool Calling - Simple: Law, Finance & Records (set 3) 9 passed1 failed

90.0%

IN79 545 OUT19 171

×

SimpleCaseInfoVictimBFCL/simple_python_160 (get_case_info): docket 2022/AL2562, court California/CA, info_type victim.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 955 OUT5 418

✓

SimpleCrimeStatuteTheftBFCL/simple_python_161 (crime_statute_lookup): jurisdiction California/CA, crime theft, detail_level detailed.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 178 OUT3 203

✓

SimpleCurrentWeatherCelsiusBFCL/simple_python_186 (current_weather_condition): city Tokyo, country Japan, measurement c/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 178 OUT1 857

✓

SimpleEnvironmentalAirQualityDaysBFCL/simple_python_199 (environmental_data.air_quality_index): location San Jose, days 3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 162 OUT1 071

✓

SimpleFelonyCrimeRecordBFCL/simple_python_156 (crime_record.get_record): case_number CA123456, county San Diego, details true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 194 OUT1 224

✓

SimpleHighestGrossingBankBFCL/simple_python_151 (highest_grossing_banks): country U.S variants, year 2020, top_n 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 186 OUT1 557

✓

SimpleHistoricalFraudCaseBFCL/simple_python_170 (law_case_search.find_historical): subject fraud, from_year 2010, to_year 2015.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 194 OUT780

✓

SimpleLawsuitCasesByCompanyYearBFCL/simple_python_177 (get_lawsuit_cases): company_name Facebook, year 2018, status all/omitted. EXPECTED MODEL MISS (kept, do not loosen): "Patent" is the case TYPE, not the status (enum is open/closed/all); status should be omitted/"all". Models often wrongly set status="Patent".1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 168 OUT1 878

✓

SimpleLawsuitInfoByCaseNumberBFCL/simple_python_182 (lawsuit_info): case_number XYZ123, year omitted/2023, location omitted/all.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 157 OUT739

✓

SimplePlantsForSlopeErosionBFCL/simple_python_194 (get_plants_for_slope): slope_type hill/steep/moderate, num_results 3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 173 OUT1 444

Tool Calling - Simple: Law, Finance & Records (set 4) 9 passed1 failed

90.0%

IN77 406 OUT11 150

✓

SimpleActDetailsAmendmentBFCL/simple_python_159 (get_act_details): act_name Criminal Law Amendment Act, amendment_year 2013.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 328 OUT629

✓

SimpleCheckFeloniesByNameBFCL/simple_python_157 (criminal_history.check_felonies): full_name John Doe, birth_date 01-01-1980, state California/CA.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 368 OUT993

✓

SimpleCourtCaseSearchNoFullTextBFCL/simple_python_169 (court_case.search): docket_number 123456, location Texas, full_text false/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 355 OUT745

✓

SimpleCurrentWeatherTempHumidityBFCL/simple_python_187 (get_current_weather): location Seattle, include_temperature true/omitted, include_humidity true/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 319 OUT767

✓

SimpleLawsuitCheckClosedStatusBFCL/simple_python_184 (lawsuit.check_case): case_id 1234, closed_status true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 341 OUT1 248

✓

SimpleLawsuitsSearchByCompanyLocationBFCL/simple_python_180 (lawsuits_search): company_name Google, location California/CA, year 2020, case_type all/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 336 OUT888

✓

SimpleMutualFundBalanceBFCL/simple_python_152 (calculate_mutual_fund_balance): investment_amount 50000, annual_yield 0.05, years 3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 366 OUT1 160

✓

SimpleSlopeGradientCoordinatesBFCL/simple_python_192 (calculate_slope_gradient): point1 [40.7128,-74.006], point2 [34.0522,-118.2437], unit degree/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 429 OUT970

✓

SimpleStateLawCaseYearRangeBFCL/simple_python_173 (law_case_search): topic land disputes, year_range [2015,2021], location New York, judicial_system state.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 377 OUT1 037

×

SimpleTopConstitutionalCasesBFCL/simple_python_174 (get_top_cases): field_of_law constitutional law, top_number 10, country China/CN.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 187 OUT2 713

Tool Calling - Simple: Law, Finance & Records (set 5) 7 passed3 failed

70.0%

IN83 441 OUT18 138

✓

SimpleCarbonFootprintLifestyleBFCL/simple_python_195 (calculate_carbon_footprint): daily_miles 20, meat_meals_per_week 3, annual_trash_weight 500, flights_per_year omitted/0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 263 OUT2 129

✓

SimpleDailyWaterIntakeWeightBFCL/simple_python_198 (calculate_daily_water_intake): weight 70, activity_level omitted/moderate, climate omitted/temperate.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 189 OUT1 933

×

SimpleFetchLawCaseByNumberBFCL/simple_python_171 (fetch_law_case_details): case_number 43403, court New York, year 2018.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 025 OUT1 205

✓

SimpleFutureValueByPresentValueBFCL/simple_python_154 (calculate_future_value): present_value 5000, annual_interest_rate 0.05, years 10, compounds_per_year omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 242 OUT1 983

✓

SimpleHumidityForecastDaysBFCL/simple_python_188 (weather.humidity_forecast): location Miami, days 7, min_humidity omitted/0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 196 OUT1 800

×

SimpleLatestCourtCaseCompaniesBFCL/simple_python_179 (find_latest_court_case): company1 Apple, company2 Samsung, country USA/omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 974 OUT1 270

×

SimpleLawsuitSearchByCountyBFCL/simple_python_183 (lawsuit_search): entity Apple, county Santa Clara County, state California/omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 951 OUT2 100

✓

SimpleLegalCaseFetchDetailsBFCL/simple_python_172 (legal_case.fetch): case_id R vs Adams, details true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 198 OUT1 554

✓

SimpleParkElevationAreaBFCL/simple_python_190 (park_information): park_name Yellowstone, information [Elevation, Area] (order-independent).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 194 OUT1 956

✓

SimpleTallestMountainsRadiusBFCL/simple_python_191 (locate_tallest_mountains): location Denver, radius 50, amount 5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 209 OUT2 208

Tool Calling - Simple: Live Answer Tool (isolated) (set 1) 1 passed

100.0%

IN4 023 OUT1 375

✓

SimpleAnswerStringLogistic1BFCL/live_simple_32-9-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L113: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 023 OUT1 375

Tool Calling - Simple: Live Answer Tool (isolated) (set 2) 1 passed

100.0%

IN3 976 OUT1 266

✓

SimpleAnswerStringLogistic2BFCL/live_simple_33-10-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L114: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 976 OUT1 266

Tool Calling - Simple: Live Answer Tool (isolated) (set 3) 1 skipped

-

IN0 OUT0

–

SimpleAnswerStringValidBFCL/live_simple_34-11-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L115: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Harness-semantic mismatch (item flaw for our runner): the query ('...Valid?') is directly answerable, and our system prompt permits answering in plain text when no tool is needed, so the model reasonably returns 0 tool calls instead of routing to the answer_string meta-tool. Not a deterministic tool-routing test. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Simple: Live Answer Tool (isolated) (set 4) 1 passed

100.0%

IN4 037 OUT2 070

✓

SimpleAnswerStringWhatIsBFCL/live_simple_35-12-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L116: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 037 OUT2 070

Tool Calling - Simple: Live Assistants, Weather & Devices (set 1) 2 passed1 failed

66.7%

IN14 737 OUT2 129

✓

SimpleChangeFoodBurgerBFCL/live_simple_20-4-0 (change_food): food_item='burger', modification_request='no onions, extra cheese'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 899 OUT462

×

SimpleThinqAirCleanHighBFCL/live_simple_40-17-0 (ThinQ_Connect): AIR_CLEAN mode, HIGH wind, monitoring on, air clean POWER_ON. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. windStrength broadened (high|strongest|strong) for "strongest wind strength". EXPECTED MODEL MISS: 27B sets airConJobMode=AIR_CLEAN but omits airCleanOperationMode=POWER_ON — the prompt asks to "start the air purifying function", so POWER_ON is the fair GT; kept strict (a real tool-use miss).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.airCleanOperationMode?.ToLower()).AnyOf(["power_on"])

Failed

IN4 939 OUT400

✓

SimpleWeatherTelAvivBFCL/live_simple_4-3-0 (get_current_weather): location='Tel Aviv, Israel', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 899 OUT1 267

Tool Calling - Simple: Live Assistants, Weather & Devices (set 10) 2 passed1 failed

66.7%

IN13 966 OUT6 283

×

SimpleChafodOrderPizzaBFCL/live_simple_23-5-1 (ChaFod): TheFod='PIZZA'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: call.ToolName == expectedTool Values: call.ToolName = "uber_eat_order" expectedTool = "ChaFod"

Failed

IN4 640 OUT3 905

✓

SimpleUberEatMcdonaldsPizzaBFCL/live_simple_29-7-2 (uber.eat.order): restaurant="McDonald's", items=[pizza], quantities=[1].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 660 OUT916

✓

SimpleWeatherShanghaiBFCL/live_simple_13-3-9 (get_current_weather): location='Shanghai, China', unit=celsius. Prompt is Chinese, copied verbatim.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 666 OUT1 462

Tool Calling - Simple: Live Assistants, Weather & Devices (set 11) 2 passed1 failed

66.7%

IN15 099 OUT4 576

✓

SimpleChafodSwitchBurgerBFCL/live_simple_24-5-2 (ChaFod): TheFod='BURGER'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 005 OUT332

✓

SimpleFindBeerSierraNevadaBFCL/live_simple_48-21-0 (find_beer): brewery='Sierra Nevada', taste=bitter, aroma=hoppy, color=pale, style=lager; defaults omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 065 OUT2 063

×

SimpleWeatherChennaiBFCL/live_simple_15-3-11 (get_current_weather): location='Chennai, Tamil Nadu', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.location?.ToLower()).AnyOf(["chennai, tamil nadu"])

Failed

IN5 029 OUT2 181

Tool Calling - Simple: Live Assistants, Weather & Devices (set 12) 3 passed

100.0%

IN15 215 OUT2 941

✓

SimpleChafodGreensSaladBFCL/live_simple_25-5-3 (ChaFod): TheFod='SALAD' (greens mapped to salad).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 030 OUT726

✓

SimpleFindBeerCreekPorterBFCL/live_simple_49-21-1 (find_beer): brewery='Creek', taste='slightly bitter', aroma=chocolatey, color=dark, style=porter; defaults omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 122 OUT850

✓

SimpleWeatherQuintanaRooBFCL/live_simple_19-3-15 (get_current_weather): location='Quintana Roo, Mexico', unit=celsius.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 063 OUT1 365

Tool Calling - Simple: Live Assistants, Weather & Devices (set 13) 1 passed1 failed

50.0%

IN9 603 OUT3 705

✓

SimpleLexListExportsV2BFCL/live_simple_30-8-0 (aws.lexv2_models.list_exports): botId='my-bot-id', botVersion='v2', sortBy=ASC or omitted, others omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 838 OUT1 720

×

SimpleWeatherLangSonBFCL/live_simple_16-3-12 (get_current_weather): location='Lang Son, Vietnam', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.location?.ToLower()).AnyOf(["lang son, vietnam"])

Failed

IN4 765 OUT1 985

Tool Calling - Simple: Live Assistants, Weather & Devices (set 14) 2 passed

100.0%

IN9 600 OUT4 675

✓

SimpleLexListExportsDescBFCL/live_simple_31-8-1 (aws.lexv2_models.list_exports): botId='B12345', botVersion='v1', sortBy=DESC, others omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 842 OUT1 877

✓

SimpleWeatherBostonBFCL/live_simple_17-3-13 (get_current_weather): location='Boston, MA', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 758 OUT2 798

Tool Calling - Simple: Live Assistants, Weather & Devices (set 15) 2 passed

100.0%

IN8 714 OUT4 574

✓

SimpleGetUserInfoBlackBFCL/live_simple_0-0-0 (get_user_info): user_id=7890, special='black'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 365 OUT675

✓

SimpleWeatherMoscowBFCL/live_simple_18-3-14 (get_current_weather): location='Moscow, Russia', unit=celsius.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 349 OUT3 899

Tool Calling - Simple: Live Assistants, Weather & Devices (set 16) 2 passed1 failed

66.7%

IN28 656 OUT11 394

✓

SimpleFetchWeatherCoordsBFCL/live_simple_39-16-0 (fetch_weather_data): latitude=37.8651, longitude=-119.5383; url/units/language defaults omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 879 OUT3 028

×

SimpleGithubStarGorillaBFCL/live_simple_1-1-0 (github_star): repos='ShishirPatil/gorilla,gorilla-llm/gorilla-cli', aligned=true.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN18 940 OUT6 538

✓

SimpleWeatherBangkokBFCL/live_simple_14-3-10 (get_current_weather): location='Bangkok, Thailand', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 837 OUT1 828

Tool Calling - Simple: Live Assistants, Weather & Devices (set 17) 3 passed

100.0%

IN13 882 OUT4 621

✓

SimpleCarbonIntensityGBBFCL/live_simple_50-22-0 (get_latest_carbon_intensity): zone='Great Britain' or 'GB'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 590 OUT1 341

✓

SimpleMultiplyThreeTwoBFCL/live_simple_47-20-0 (multiply): a=3, b=2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 592 OUT582

✓

SimpleWeatherOpenMeteoMariposaBFCL/live_simple_38-15-0 (get_current_weather, open-meteo variant): location='Mariposa, CA', unit=fahrenheit; url default omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 700 OUT2 698

Tool Calling - Simple: Live Assistants, Weather & Devices (set 2) 3 passed

100.0%

IN14 695 OUT3 437

✓

SimpleChangeFoodChickenSpicyBFCL/live_simple_21-4-1 (change_food): food_item='chicken dish', modification_request='extra spicy'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 920 OUT1 452

✓

SimpleThinqAirDryMidBFCL/live_simple_41-17-1 (ThinQ_Connect): AIR_DRY mode, MID wind (or omitted), all else optional. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. windStrength broadened (mid|medium) for "medium wind strength"; air-clean off-state accepts off/power_off.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 889 OUT511

✓

SimpleWeatherDivinopolisBFCL/live_simple_5-3-1 (get_current_weather): location='Divinopolis, MG' (verbatim), unit=fahrenheit or omitted. Prompt is Portuguese, copied verbatim.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 886 OUT1 474

Tool Calling - Simple: Live Assistants, Weather & Devices (set 3) 3 passed

100.0%

IN14 694 OUT6 285

✓

SimpleParseAnswerNoDefineBFCL/live_simple_36-13-0 (parseAnswer): answer='The text does not define logistic regression, therefore I cannot answer this question.'1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 891 OUT1 910

✓

SimpleThinqKoreanTarget22BFCL/live_simple_42-17-2 (ThinQ_Connect): coolTargetTemperature=22 required; mode COOL and wind MID optional. Prompt is Korean, copied verbatim. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. Prompt is dehumidify (제습) so GT's optional airConJobMode=COOL is a source oddity — accept cool/dehumidify/air_dry (optional). windStrength MID broadened (mid|medium). The "22도" target-temp is accepted in either coolTargetTemperature or targetTemperature (the two are genuinely ambiguous for "target temperature").1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 920 OUT1 312

✓

SimpleWeatherSanFranciscoBFCL/live_simple_6-3-2 (get_current_weather): location='San Francisco, CA', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 883 OUT3 063

Tool Calling - Simple: Live Assistants, Weather & Devices (set 4) 3 passed

100.0%

IN14 577 OUT5 431

✓

SimpleParseAnswerNoDefineValidityBFCL/live_simple_37-14-0 (parseAnswer): answer='The text does not define logistic regression, therefore I cannot answer this question.'1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 873 OUT1 725

✓

SimpleThinqCoolHighWindBFCL/live_simple_43-17-3 (ThinQ_Connect): HIGH wind required; COOL mode and coolTargetTemperature=24 optional. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. air-clean off-state broadened (power_off|off) — the model files the unrequested clean-mode as "off".1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 866 OUT1 151

✓

SimpleWeatherRigaBFCL/live_simple_7-3-3 (get_current_weather): location='Riga, Latvia'; unit fahrenheit/celsius/omitted all accepted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 838 OUT2 555

Tool Calling - Simple: Live Assistants, Weather & Devices (set 5) 3 passed

100.0%

IN15 192 OUT2 358

✓

SimpleThinqTurnedOnCoolBFCL/live_simple_44-18-0 (ThinQ_Connect): airConOperationMode=POWER_ON required; COOL currentJobMode optional. NOTE: tool flattened to nullable top-level params (see struct comment) so a valid partial call is one tool call, not a forced retry. windStrength/airClean off-state broadened to synonyms (mid|medium, stop|off|power_off).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 041 OUT501

✓

SimpleUberComfortAddisonBFCL/live_simple_2-2-0 (uber.ride): loc='2020 Addison Street, Berkeley, CA, USA', type=comfort, time=600.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 103 OUT340

✓

SimpleWeatherLondonBFCL/live_simple_8-3-4 (get_current_weather): location='London, UK' or 'London, England'; unit fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 048 OUT1 517

Tool Calling - Simple: Live Assistants, Weather & Devices (set 6) 1 passed2 failed

33.3%

IN15 224 OUT3 765

×

SimpleThinqCoolAirCleanStartBFCL/live_simple_45-18-1 (ThinQ_Connect): airCleanOperationMode=START, airConOperationMode=POWER_ON, coolTargetTemperature=20. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. airCleanOperationMode "air cleaning on" broadened (start|on|power_on). airConOperationMode=POWER_ON and coolTargetTemperature=20 kept strict — both fair GT. EXPECTED MODEL MISS: 27B gets job-mode + air-clean START + temp, but omits airConOperationMode=POWER_ON.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.airConOperationMode?.ToLower()).AnyOf(["power_on"])

Failed

IN5 092 OUT787

✓

SimpleUberPlusBakerBFCL/live_simple_3-2-1 (uber.ride): loc='221B Baker Street, Berkeley, CA, USA', type=plus, time=600.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 105 OUT767

×

SimpleWeatherHyderabadBFCL/live_simple_9-3-5 (get_current_weather): location='Hyderabad, India', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.location?.ToLower()).AnyOf(["hyderabad, india"])

Failed

IN5 027 OUT2 211

Tool Calling - Simple: Live Assistants, Weather & Devices (set 7) 3 passed

100.0%

IN15 244 OUT3 831

✓

SimpleThinqFlatCoolCleanBFCL/live_simple_46-19-0 (ThinQ_Connect, flat params): COOL, MID, monitoring on, clean START, POWER_ON, power-save off, target 24C.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 144 OUT1 964

✓

SimpleUberPlusVietnameseBFCL/live_simple_26-6-0 (uber.ride): loc Vietnamese/English form, type=plus, time=10 (minutes). Prompt is Vietnamese, copied verbatim.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 067 OUT553

✓

SimpleWeatherYosemiteMariposaBFCL/live_simple_10-3-6 (get_current_weather): location='Mariposa, CA', unit=celsius.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 033 OUT1 314

Tool Calling - Simple: Live Assistants, Weather & Devices (set 8) 1 passed2 failed

33.3%

IN15 065 OUT2 479

×

SimpleThinqCoolScheduledBFCL/live_simple_51-23-0 (ThinQ_Connect): COOL, HIGH wind, POWER_ON, power-save on, target 22C, start in 1h30m. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. airConOperationMode=POWER_ON kept strict ("Start in 1h30m" schedules a power-on). EXPECTED MODEL MISS: 27B gets job-mode COOL + wind + power-save + temp + schedule, but omits airConOperationMode=POWER_ON.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.airConOperationMode?.ToLower()).AnyOf(["power_on"])

Failed

IN5 096 OUT617

×

SimpleUberEatPitadaBFCL/live_simple_27-7-0 (uber.eat.order): restaurant='uber pitada', items=[burgers, chicken wings], quantities=[5, 6].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.items[0].ToLower()).AnyOf(["burgers"])

Failed

IN5 004 OUT497

✓

SimpleWeatherNaplesBFCL/live_simple_11-3-7 (get_current_weather): location='Naples, FL' or 'Naples, Florida'; unit fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 965 OUT1 365

Tool Calling - Simple: Live Assistants, Weather & Devices (set 9) 1 passed2 failed

33.3%

IN14 075 OUT7 505

×

SimpleChafodWhopperBurgerBFCL/live_simple_22-5-0 (ChaFod): TheFod='BURGER' (Whopper mapped to burger).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: call.ToolName == expectedTool Values: call.ToolName = "uber_eat_order" expectedTool = "ChaFod"

Failed

IN4 657 OUT1 230

×

SimpleUberEatKfcChineseBFCL/live_simple_28-7-1 (uber.eat.order): KFC order, 4 items, quantities [10,50,30,90]. Prompt is Chinese, copied verbatim.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.items[0].ToLower()).AnyOf(["麦辣鸡腿堡"])

Failed

IN4 720 OUT3 599

✓

SimpleWeatherNewYorkBFCL/live_simple_12-3-8 (get_current_weather): location='New York, NY', unit=celsius.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 698 OUT2 676

Tool Calling - Simple: Live Media, Weather & Data (set 2) 9 passed1 failed

90.0%

IN76 082 OUT15 550

✓

SimpleCreditQuoteAutoCreditoBFCL/live_simple_67-31-0 (obtener_cotizacion_de_creditos): monto=1000000, plazo=12, producto='auto', año=2024, enganche=0.2.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 698 OUT3 360

✓

SimpleCurrentLocDmsBFCL/live_simple_87-48-0 (get_current_loc): coordinate_format='DMS', include_altitude=true, timeout default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 583 OUT1 970

✓

SimpleGetMoviesMumbaiTodayBFCL/live_simple_59-28-0 (get_movies): city='Mumbai', optionals default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 551 OUT1 838

✓

SimplePlayArtistKaarijaTracksBFCL/live_simple_93-54-0 (play_artist): artist_name='Käärijä'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 566 OUT220

✓

SimpleRetrieveAnalysisSuperfancyBFCL/live_simple_69-33-0 (analysis_api.AnalysisApi.retrieve_analysis): project/component/vulnerability UUIDs.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 681 OUT456

✓

SimpleSensorAlertsPerPage10BFCL/live_simple_70-34-0 (get_sensor_alerts): perPage=10, sensorSerial='Q3CC-CRT3-SZ2G', rest omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 629 OUT1 834

✓

SimpleSitefinityNewsItemJulian12BFCL/live_simple_81-42-0 (sitefinity_create_contentitem): ContentItem='NewsItem', Title='Julian is testing12', rest omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 632 OUT306

✓

SimpleTodoAddShopping9pmB25BFCL/live_simple_55-25-1 (todo): type='add', content='go for shopping at 9 pm'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 575 OUT318

×

SimpleTodoDeleteTodoRandomBFCL/live_simple_54-25-0 (todo): type='delete', content='todo random'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.content?.ToLower()).AnyOf(["todo random"])

Failed

IN7 577 OUT1 792

✓

SimpleWeatherGetLondonUkBFCL/live_simple_77-38-0 (weather.get): city='London', country='GB' (from UK), units/forecast default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 590 OUT3 456

Tool Calling - Simple: Live Media, Weather & Data (set 3) 9 passed1 failed

90.0%

IN71 617 OUT8 992

×

SimpleExtractMillennialsAppleBFCL/live_simple_71-35-0 (extract_parameters_v1): demographics=[millennials], targets=[brand:Apple], metrics=[view], min_date=2022-07-01. NOTE: targets @param clarified to '<attribute>:<entity>' order (was ambiguous 'target:target_type', which made the model emit the reversed 'Apple:brand'); GT 'brand:Apple' kept strict.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.demographics[0]?.ToLower()).AnyOf(["millennials"])

Failed

IN7 178 OUT2 165

✓

SimpleFetchPersonId4BFCL/live_simple_91-52-0 (fetchPerson): id=4.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 137 OUT1 477

✓

SimpleLogFoodChaiTeaBFCL/live_simple_88-49-0 (log_food): food_name='chai tea', portion_amount=16.0, portion_unit='ounces', meal_name='snack'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 207 OUT363

✓

SimplePlaySongChaChaChaBFCL/live_simple_94-55-0 (play_song): artist_name='Käärijä', song_name='Cha Cha Cha'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 155 OUT250

✓

SimpleSendEmailAndyForecastBFCL/live_simple_78-39-0 (send_email): to='andy@gorilla.ai', subject='Sales Forecast Request', body set, cc/bcc/attachments omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 203 OUT557

✓

SimpleSitefinityNewsJulianTestingBFCL/live_simple_82-43-0 (sitefinity_create_contentitem): ContentItem='News', Title='Julian is testing', rest omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 171 OUT372

✓

SimpleTemperatureParisFranceBFCL/live_simple_90-51-0 (temperature): location='Paris, France', units/time default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 129 OUT1 443

✓

SimpleTodoManagerAddSleepBFCL/live_simple_57-26-1 (todo_manager.handle_action): type='add', content='go to sleep at 9 pm'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 142 OUT335

✓

SimpleTodoManagerDeleteGymBFCL/live_simple_56-26-0 (todo_manager.handle_action): type='delete', content='go to gym'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 132 OUT431

✓

SimpleTranslateEnglishFrenchBFCL/live_simple_76-37-0 (language_translator.translate): source='en', target='fr', text='What is your name?'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 163 OUT1 599

Tool Calling - Simple: Live Media, Weather & Data (set 4) 10 passed

100.0%

IN53 287 OUT9 242

✓

SimpleOpenMeteoBerkeleyBFCL/live_simple_85-46-0 (open_meteo_api.fetch_weather_data): coordinates=[37.8715,-122.273], units default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 335 OUT3 169

✓

SimpleSearchHotelsSanDiegoBFCL/live_simple_79-40-0 (search_hotels): location='San Diego, CA'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 384 OUT2 093

✓

SimpleSitefinityNewsSleepProductivityBFCL/live_simple_83-44-0 (sitefinity_create_contentitem): full news article with all metadata.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 478 OUT654

✓

SimpleSortArrayAscendingBFCL/live_simple_86-47-0 (sort_array): array=[1,54,3,1,76,2343,21], reverse default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 354 OUT1 162

✓

SimpleTodoAddGoToGoaBFCL/live_simple_64-29-4 (todo): type='add', content='go to Goa'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 281 OUT352

✓

SimpleTodoAddGymTomorrowBFCL/live_simple_61-29-1 (todo): type='add', content='go to gym tomorrow.'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 280 OUT398

✓

SimpleTodoAddHiCharlieBFCL/live_simple_65-29-5 (todo): type='add', content='Hi charlie'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 291 OUT625

✓

SimpleTodoAddSleep9pmBFCL/live_simple_63-29-3 (todo): type='add', content='go to sleep at 9 pm'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 287 OUT265

✓

SimpleTodoCompleteFinancialReportBFCL/live_simple_60-29-0 (todo): type='complete', content='Submit monthly financial report'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 304 OUT274

✓

SimpleTodoDeleteRaviBFCL/live_simple_62-29-2 (todo): type='delete', content='ravi'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 293 OUT250

Tool Calling - Simple: Live Movies, Music & Booking (set 1) 3 passed

100.0%

IN14 597 OUT1 762

✓

SimpleMoviesTarantinoWhitakerBFCL/live_simple_208-117-0 (Movies_3_FindMovies): directed_by='Quentin Tarantino', cast='Duane Whitaker', genre dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 886 OUT1 250

✓

SimplePlaySongRoarBFCL/live_simple_230-121-0 (play_song): song_id='wjeiruhuq_roar', artist='Katy Perry', quality='high', device_id omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 883 OUT323

✓

SimpleSetVolume20BFCL/live_simple_242-126-0 (set_volume): volume=20.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 828 OUT189

Tool Calling - Simple: Live Movies, Music & Booking (set 10) 3 passed

100.0%

IN15 616 OUT2 577

✓

SimpleHandoverPackageArrivalBFCL/live_simple_257-137-1 (answer_question): statement=package-arrival-time question, urgency/language default. System instructs HANDOVER on question. NOTE: no collider loop observed (single clean call, sibling tests pass); statement is free text -> substring-intent of the key phrase (model drops "Can you tell me").1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 217 OUT747

✓

SimpleMoviesNiteshDirectorBFCL/live_simple_217-117-9 (Movies_3_FindMovies): directed_by='Nitesh Tiwari', genre/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 186 OUT1 041

✓

SimpleTtsChineseMaleBFCL/live_simple_227-118-1 (text_to_speech.convert): text=Chinese phrase, language='zh-CN', gender='male', format='wav', speed default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 213 OUT789

Tool Calling - Simple: Live Movies, Music & Booking (set 11) 3 passed

100.0%

IN14 956 OUT5 366

✓

SimpleMoviesColaizzoWatkinsBFCL/live_simple_218-117-10 (Movies_3_FindMovies): directed_by='Paul Downs Colaizzo', cast='Michaela Watkins', genre dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 045 OUT3 485

✓

SimpleRequestsGetBuProductsBFCL/live_simple_229-120-0 (requests.get): anchor='user'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 954 OUT1 066

✓

SimpleTtsPrettyGirlDefaultBFCL/live_simple_228-119-0 (text_to_speech.convert): text='I am a pretty girl', language/gender/speed default.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 957 OUT815

Tool Calling - Simple: Live Movies, Music & Booking (set 12) 2 passed1 broken

66.7%

IN14 662 OUT65 055

✓

SimpleBookFlightParisBFCL/live_simple_233-123-0 (book_flight): departure_location='Paris, France', departure_time='12/03/2023 15:00', return_time omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 923 OUT552

✓

SimpleGetVersionBFCL/live_simple_247-129-0 (version_api.VersionApi.get_version): no arguments.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 835 OUT2 146

!

SimpleMoviesSimonCurtisDramaBFCL/live_simple_219-117-11 (Movies_3_FindMovies): directed_by='Simon Curtis', genre='Drama', cast='McKinley Belcher III'.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Broken

IN4 904 OUT62 357

Tool Calling - Simple: Live Movies, Music & Booking (set 13) 3 passed

100.0%

IN15 199 OUT3 892

✓

SimpleAclRetrieveProjectsBFCL/live_simple_248-130-0 (acl_api.AclApi.retrieve_projects): uuid='12', excludeInactive=true, onlyRoot=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 054 OUT1 885

✓

SimpleBookFlightNewYorkToParisBFCL/live_simple_234-123-1 (book_flight): departure_location='New York, NY', departure_time='20/06/2022 17:00', return_time omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 104 OUT723

✓

SimpleMoviesMackendrickThrillerBFCL/live_simple_220-117-12 (Movies_3_FindMovies): directed_by='Alexander Mackendrick', genre='Thriller', cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 041 OUT1 284

Tool Calling - Simple: Live Movies, Music & Booking (set 14) 3 passed

100.0%

IN14 821 OUT3 074

✓

SimpleBookFlightJfkBFCL/live_simple_245-127-0 (book_flight): departure_location='JFK', departure_date='16/03/2024', departure_time='14:00'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 995 OUT486

✓

SimpleChatWithBunnyBFCL/live_simple_249-131-0 (chat_with_friend): user_message='Hey bunny, How are you doing'. NOTE: no collider loop observed (single clean call, sibling tests pass); user_message is a free-text passthrough -> substring-intent (model may drop the "Hey bunny," greeting).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 910 OUT1 555

✓

SimpleMoviesJamesGrayBFCL/live_simple_221-117-13 (Movies_3_FindMovies): directed_by='James Gray', genre/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 916 OUT1 033

Tool Calling - Simple: Live Movies, Music & Booking (set 15) 2 passed

100.0%

IN9 291 OUT1 965

✓

SimpleBookFlightNewYorkMarchBFCL/live_simple_246-128-0 (book_flight): departure_location='New York, NY', departure_time='15/03/2024'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 642 OUT374

✓

SimpleMoviesVanOrmanBrownBFCL/live_simple_222-117-14 (Movies_3_FindMovies): directed_by='Thurop Van Orman', cast='Sterling K. Brown', genre dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 649 OUT1 591

Tool Calling - Simple: Live Movies, Music & Booking (set 16) 1 passed1 skipped

100.0%

IN4 598 OUT1 798

✓

SimpleMoviesImaginativeFantasyBFCL/live_simple_223-117-15 (Movies_3_FindMovies): genre='Fantasy' ('Imaginative fiction'), director/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 598 OUT1 798

–

SimpleRescheduleGymSessionBFCL/live_simple_231-122-0 (reschedule_event): event_identifier='456123', new_datetime='2022-10-30T16:30:00Z'.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: source flaw: relative-date arithmetic is inconsistent — prompt states 'Monday 2022-10-20' (that date is actually a Thursday) and the GT 'next Thursday' resolves to 2022-10-30 (a Sunday), so the exact new_datetime cannot be derived deterministically.

Skipped

IN0 OUT0

Tool Calling - Simple: Live Movies, Music & Booking (set 17) 2 passed

100.0%

IN9 266 OUT3 418

✓

SimpleMoviesTimBurtonDeppBFCL/live_simple_224-117-16 (Movies_3_FindMovies): directed_by='Tim Burton', genre='Offbeat', cast='Johnny Depp'.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 589 OUT2 489

✓

SimpleRescheduleMedAppointmentBFCL/live_simple_232-122-1 (reschedule_event): event_identifier='med123', new_datetime='2021-04-15T13:45:00Z'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 677 OUT929

Tool Calling - Simple: Live Movies, Music & Booking (set 18) 2 passed

100.0%

IN8 954 OUT1 286

✓

SimpleMoviesKirillComedyDramaBFCL/live_simple_225-117-17 (Movies_3_FindMovies): directed_by='Kirill Mikhanovsky', genre='Comedy-drama', cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 512 OUT1 070

✓

SimpleSetVolume70BFCL/live_simple_244-126-2 (set_volume): volume=70.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 442 OUT216

Tool Calling - Simple: Live Movies, Music & Booking (set 2) 3 passed

100.0%

IN14 642 OUT2 355

✓

SimpleMoviesLeitchTuisanoBFCL/live_simple_209-117-1 (Movies_3_FindMovies): directed_by='David Leitch', cast='Lori Pelenise Tuisano', genre dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 911 OUT1 809

✓

SimpleSetVolume30BFCL/live_simple_243-126-1 (set_volume): volume=30.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 864 OUT240

✓

SimpleSpotifyFriendsMarshmelloBFCL/live_simple_235-124-0 (play_spotify_song): query='track:Friends artist:Marshmello', shuffle/volume default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 867 OUT306

Tool Calling - Simple: Live Movies, Music & Booking (set 3) 3 passed

100.0%

IN15 238 OUT3 144

✓

SimpleFlightStatusIndigo6E123BFCL/live_simple_251-133-0 (flight.status.check): flight_id='6E123', passenger_name='Karan', ticket_number='IND4567'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 102 OUT1 441

✓

SimpleMoviesDramaBFCL/live_simple_210-117-2 (Movies_3_FindMovies): genre='Drama', director/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 072 OUT1 264

✓

SimpleSpotifyDilNu124BFCL/live_simple_236-124-1 (play_spotify_song): query='track:Dil Nu artist:Maninder Buttar', shuffle/volume default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 064 OUT439

Tool Calling - Simple: Live Movies, Music & Booking (set 4) 3 passed

100.0%

IN15 214 OUT2 570

✓

SimpleFlightStatusIndigoAirlineBFCL/live_simple_252-134-0 (flight.status.check): airlines='Indigo', person_name='Karan', ticket_number='IND4567'. NOTE: tool doc says "if only one name mentioned ... use LNU", so "Karan LNU" is equivalent-correct per the documented convention -> accept both.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 096 OUT674

✓

SimpleMoviesComedyCordenBFCL/live_simple_211-117-3 (Movies_3_FindMovies): genre='Comedy' (prompt 'Comic'), cast='James Corden', director dontcare. NOTE: args are correct; single-call contract stays strict. The 27B occasionally emits a 2nd call (over-call) -> legit intermittent fail (kept as signal).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 083 OUT1 223

✓

SimpleSpotifyWreckingBallBFCL/live_simple_237-125-0 (play_spotify_song): query='track:Wrecking Ball artist:Miley Cyrus', device_id/play default. NOTE: GT accepts only the structured 'track:.. artist:..' form (kept strict). The 27B occasionally drops the format (e.g. "wrecking ball Miley Cyrus") -> legit intermittent miss (kept as signal).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 035 OUT673

Tool Calling - Simple: Live Movies, Music & Booking (set 5) 3 passed

100.0%

IN15 131 OUT1 897

✓

SimpleMoviesComedyEdgarWrightBFCL/live_simple_212-117-4 (Movies_3_FindMovies): directed_by='Edgar Wright', genre='Comedy', cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 028 OUT1 148

✓

SimpleSpotifyDilNu125BFCL/live_simple_238-125-1 (play_spotify_song): query='track:Dil Nu artist:Maninder Buttar', device_id/play default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 037 OUT407

✓

SimpleUberEatsBigMacBFCL/live_simple_250-132-0 (uber.eat.order): restaurants='McDonald's', items=[Big Mac, McFlurry], quantities=[2,1].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 066 OUT342

Tool Calling - Simple: Live Movies, Music & Booking (set 6) 2 passed1 failed

66.7%

IN15 156 OUT4 560

✓

SimpleMoviesOffbeatTimBurtonBFCL/live_simple_213-117-5 (Movies_3_FindMovies): directed_by='Tim Burton', genre='Offbeat', cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 047 OUT2 624

×

SimpleSpotifyShapeOfYouBFCL/live_simple_239-125-2 (play_spotify_song): query='track:Shape of You artist:Ed Sheeran' or 'Shape of You'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.play.AnyOf([null, true]) Values: args.play = false

Failed

IN5 042 OUT965

✓

SimpleUberEatsBurgersWingsBFCL/live_simple_253-135-0 (uber.eat.order): restaurants='McDonald's', items=[burgers, chicken wings], quantities=[5,6].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 067 OUT971

Tool Calling - Simple: Live Movies, Music & Booking (set 7) 3 passed

100.0%

IN14 960 OUT3 340

✓

SimpleMoviesComicNiteshTiwariBFCL/live_simple_214-117-6 (Movies_3_FindMovies): directed_by='Nitesh Tiwari', genre='Comedy' or 'Comedy-drama', cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 039 OUT1 884

✓

SimpleSpotifyBabySharkBFCL/live_simple_240-125-3 (play_spotify_song): query='Baby Shark' (artist unknown).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 949 OUT420

✓

SimpleUserMandatesPendingBFCL/live_simple_254-136-0 (user.mandates): user_id='U123', status='pending'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 972 OUT1 036

Tool Calling - Simple: Live Movies, Music & Booking (set 8) 3 passed

100.0%

IN14 898 OUT3 560

✓

SimpleMoviesFantasyTonightBFCL/live_simple_215-117-7 (Movies_3_FindMovies): genre='Fantasy', director/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 968 OUT1 589

✓

SimpleSpotifyJohnnyPapaBFCL/live_simple_241-125-4 (play_spotify_song): query='Johnny Johnny Yes Papa' (artist unknown).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 945 OUT856

✓

SimpleUserMandatesActiveBFCL/live_simple_255-136-1 (user.mandates): user_id='U123456', status='active' or omitted (default).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 985 OUT1 115

Tool Calling - Simple: Live Movies, Music & Booking (set 9) 2 passed1 failed

66.7%

IN15 569 OUT3 987

×

SimpleHandoverComplaintStatusBFCL/live_simple_256-137-0 (answer_question): statement=user complaint-status question, urgency/language default. System instructs HANDOVER on question.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.statement?.ToLower()).AnyOf(["i have had submit a compaint and i like to check its status"])

Failed

IN5 210 OUT750

✓

SimpleMoviesActionLeitchKingBFCL/live_simple_216-117-8 (Movies_3_FindMovies): directed_by='David Leitch', genre='Action', cast='Alex King'.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 176 OUT2 436

✓

SimpleTtsPrettyGirlSpanishBFCL/live_simple_226-118-0 (text_to_speech.convert): text='I am a pretty girl' or Spanish translation, language='es-ES', gender/format/speed default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 183 OUT801

Tool Calling - Simple: Live Services, Weather & Commands (set 1) 4 passed

100.0%

IN18 934 OUT4 311

✓

SimpleExtractChesterJaneBFCL/live_simple_189-114-0 (extractor.extract_information): [Chester,42],[Jane,43] (Jane a year older).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 780 OUT701

✓

SimpleServiceIdIroningUnitIntBFCL/live_simple_174-100-0 (get_service_id): ironing service -> service_id=2, unit omitted/1. Thai prompt.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 727 OUT854

✓

SimpleWeatherNewYorkBFCL/live_simple_193-116-1 (Weather_1_GetWeather): city=New York, date=2023-03-08.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 713 OUT1 278

✓

SimpleWeatherPacificaBFCL/live_simple_192-116-0 (Weather_1_GetWeather): city=Pacifica, date=2023-04-11.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 714 OUT1 478

Tool Calling - Simple: Live Services, Weather & Commands (set 10) 7 passed2 failed

77.8%

IN52 330 OUT14 465

✓

SimpleCmdDockerPsBFCL/live_simple_172-99-6 (cmd_controller.execute): is Docker running -> 'docker ps'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 573 OUT654

✓

SimpleCmdDockerStartBFCL/live_simple_168-99-2 (cmd_controller.execute): start Docker -> 'docker start'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 573 OUT482

✓

SimpleCmdEchoDateBFCL/live_simple_171-99-5 (cmd_controller.execute): today's date -> 'echo %date%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 582 OUT1 141

✓

SimpleCmdEchoTimeBFCL/live_simple_166-99-0 (cmd_controller.execute): current time -> 'echo %time%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 594 OUT1 839

×

SimpleCmdListCDriveBFCL/live_simple_170-99-4 (cmd_controller.execute): list C drive -> 'dir C:\'.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 418 OUT6 118

×

SimpleCmdRunPythonScriptBFCL/live_simple_169-99-3 (cmd_controller.execute): run python script with venv interpreter.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.command?.ToLower()).AnyOf(["d:/playground/pc_contoller/env/scripts/python.exe d:/playground/pc_contoller/test.py"])

Failed

IN4 661 OUT339

✓

SimpleCmdStartCalcBFCL/live_simple_173-99-7 (cmd_controller.execute): open calculator -> 'start calc'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 575 OUT328

✓

SimpleCmdTasklistCodeBFCL/live_simple_167-99-1 (cmd_controller.execute): is VS Code running -> tasklist filter.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 606 OUT2 624

✓

SimpleProvidersChonburiDateRangeBFCL/live_simple_186-111-0 (get_service_providers): ironing, Chonburi, rating>=4.5, 12/03/2024 16:00-18:00.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 748 OUT940

Tool Calling - Simple: Live Services, Weather & Commands (set 11) 3 passed1 failed

75.0%

IN19 699 OUT8 299

✓

SimpleItemsCircleLogosBFCL/live_simple_162-96-1 (get_items): 4 circle-logo copyright items.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 826 OUT1 366

✓

SimpleItemsLegoMovieBFCL/live_simple_163-96-2 (get_items): 7 Lego-movie copyright items.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 921 OUT979

✓

SimpleItemsSuperheroLogosBFCL/live_simple_161-96-0 (get_items): 12 superhero-logo copyright items.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 083 OUT3 064

×

SimpleProvidersChonburiNoLateBFCL/live_simple_187-112-0 (get_service_providers): ironing, Chonburi, rating>=4.5, 12/03/2024, never late. NOTE: the prompt gives a time range "16:00 - 18:00", so end_available_date is expected; the 27B occasionally omits it (~1/3 of runs) -> a legit model-reliability miss, not an assertion flaw.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.end_available_date ?? "").AnyOfContains(["2024-03-12", "2024-12-03"])

Failed

IN4 869 OUT2 890

Tool Calling - Simple: Live Services, Weather & Commands (set 12) 2 passed

100.0%

IN9 434 OUT2 131

✓

SimpleCopyrightCocaColaBFCL/live_simple_164-97-0 (get_copyright_info): Coca-Cola logo, The Coca-Cola Company, confidence 100.0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 863 OUT688

✓

SimpleProfessionalCleaningRating2BFCL/live_simple_183-108-0 (getDataForProfessional): cleaning in Bangkok, rating>=2.0 -> avg_rating=2.0, province_id=1, service_id=1.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 571 OUT1 443

Tool Calling - Simple: Live Services, Weather & Commands (set 13) 3 passed

100.0%

IN13 678 OUT4 705

✓

SimpleMpnSpecs3pak7BFCL/live_simple_191-115-1 (raptor.mpn.specs): identifier=3pak7, MPN, include_images omitted/false.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 549 OUT1 186

✓

SimpleMpnSpecsAchdImagesBFCL/live_simple_190-115-0 (raptor.mpn.specs): identifier=ACHD, MPN, include_images=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 569 OUT1 609

✓

SimpleProfessionalCleaningRating4BFCL/live_simple_184-109-0 (getDataForProfessional): cleaning in Bangkok, rating>4 -> service_id=1, province_id=1, rating=4.0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 560 OUT1 910

Tool Calling - Simple: Live Services, Weather & Commands (set 14) 2 passed

100.0%

IN9 752 OUT1 750

✓

SimpleExtractChineseNamesBFCL/live_simple_165-98-0 (extractor.extract_information): Chinese text -> [李雷,18],[李丽,21] (sister 3 yrs older).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 873 OUT728

✓

SimpleProfessionalIroningChiangMaiBFCL/live_simple_188-113-0 (getDataForProfessional): ironing in Chiang Mai, Sankampang/Ton Pao, rating 4.5.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 879 OUT1 022

Tool Calling - Simple: Live Services, Weather & Commands (set 2) 3 passed

100.0%

IN13 110 OUT3 708

✓

SimpleServiceIdIroningUnitSessionBFCL/live_simple_175-101-0 (get_service_id): ironing -> service_id=2, unit omitted/session. Thai prompt.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 353 OUT800

✓

SimpleWeatherMartinezBFCL/live_simple_194-116-2 (Weather_1_GetWeather): city=Martinez, date=2023-04-25.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 387 OUT1 163

✓

SimpleWeatherPaloAltoBFCL/live_simple_195-116-3 (Weather_1_GetWeather): city=Palo Alto, date=2023-04-25.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 370 OUT1 745

Tool Calling - Simple: Live Services, Weather & Commands (set 3) 3 passed

100.0%

IN13 072 OUT5 374

✓

SimpleServiceIdIroningUnitQtyBFCL/live_simple_176-102-0 (get_service_id): ironing -> service_id=2, unit omitted/1. Thai prompt.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 339 OUT465

✓

SimpleWeatherAlamedaWednesdayBFCL/live_simple_196-116-4 (Weather_1_GetWeather): city=Alameda, upcoming Wednesday (today Mon 2023-04-25) -> 2023-04-27.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 369 OUT2 965

✓

SimpleWeatherStinsonBeachBFCL/live_simple_197-116-5 (Weather_1_GetWeather): city=Stinson Beach, date=2023-04-05.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 364 OUT1 944

Tool Calling - Simple: Live Services, Weather & Commands (set 4) 3 passed1 failed

75.0%

IN19 777 OUT6 774

✓

SimpleServiceIdCleaningBFCL/live_simple_177-103-0 (get_service_id): cleaning -> service_id=1, unit omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 325 OUT513

✓

SimpleServiceIdIroningBFCL/live_simple_178-103-1 (get_service_id): ironing -> service_id=2, unit omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 327 OUT1 019

✓

SimpleWeatherHealdsburgBFCL/live_simple_198-116-6 (Weather_1_GetWeather): city=Healdsburg, date=2023-03-02.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 380 OUT2 052

×

SimpleWeatherMarshallBFCL/live_simple_199-116-7 (Weather_1_GetWeather): city=Marshall (prompt 'Marshall, MN'), date=2023-03-05. NOTE: GT city is bare "Marshall"; model faithfully copies the "Marshall, MN" disambiguator from the prompt -> substring-intent (equivalent-correct).1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN6 745 OUT3 190

Tool Calling - Simple: Live Services, Weather & Commands (set 5) 3 passed

100.0%

IN13 209 OUT3 255

✓

SimpleServiceIdIroningChiangMaiBFCL/live_simple_179-104-0 (get_service_id): ironing in Chiang Mai -> service_id=2, province_id=2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 374 OUT742

✓

SimpleWeatherCampbellBFCL/live_simple_201-116-9 (Weather_1_GetWeather): city=Campbell, date=2023-03-04.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 431 OUT1 238

✓

SimpleWeatherFremontBFCL/live_simple_200-116-8 (Weather_1_GetWeather): city=Fremont, date=2023-03-01.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 404 OUT1 275

Tool Calling - Simple: Live Services, Weather & Commands (set 6) 3 passed

100.0%

IN13 396 OUT4 207

✓

SimpleServiceIdIroningBangkokRatingBFCL/live_simple_180-105-0 (get_service_id): ironing in Bangkok, rating>=4 -> service_id=2, province_id=1, rating=4.0.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 468 OUT1 051

✓

SimpleWeatherFosterCityBFCL/live_simple_202-116-10 (Weather_1_GetWeather): city=Foster City, date=2023-04-25.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 461 OUT1 920

✓

SimpleWeatherWashingtonBFCL/live_simple_203-116-11 (Weather_1_GetWeather): city=Washington (prompt 'Washington, DC'), date=2023-03-01. NOTE: GT city is bare "Washington"; model faithfully copies the "Washington, DC" disambiguator from the prompt -> substring-intent (equivalent-correct).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 467 OUT1 236

Tool Calling - Simple: Live Services, Weather & Commands (set 7) 3 passed

100.0%

IN13 721 OUT5 297

✓

SimpleServiceIdIroningChatuchakBFCL/live_simple_181-106-0 (get_service_id): ironing, Bangkok(Chatuchak), rating>=4.5 -> service_id=2, province_id omitted/1, district_name=Chatuchak, rating=4.5.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 585 OUT980

✓

SimpleWeatherBerkeleySaturdayBFCL/live_simple_205-116-13 (Weather_1_GetWeather): city=Berkeley, upcoming Saturday (today Tue 2023-04-25) -> 2023-04-29.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 564 OUT1 902

✓

SimpleWeatherRutherfordBFCL/live_simple_204-116-12 (Weather_1_GetWeather): city=Rutherford (prompt 'Rutherford, NJ'), date=2023-04-22. NOTE: GT city is bare "Rutherford"; model faithfully copies the "Rutherford, NJ" disambiguator from the prompt -> substring-intent (equivalent-correct).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 572 OUT2 415

Tool Calling - Simple: Live Services, Weather & Commands (set 8) 3 passed

100.0%

IN13 820 OUT6 663

✓

SimpleProvidersKhlongSamWaBFCL/live_simple_182-107-0 (get_service_providers): ironing, Bangkok, Khlong Sam Wa / Sai Kong, rating>=4.5.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 639 OUT1 724

✓

SimpleWeatherLondonBFCL/live_simple_206-116-14 (Weather_1_GetWeather): city=London (prompt 'London, England'), date=2023-03-05.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 591 OUT3 306

✓

SimpleWeatherSacramentoBFCL/live_simple_207-116-15 (Weather_1_GetWeather): city=Sacramento, date=2023-04-22.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 590 OUT1 633

Tool Calling - Simple: Live Services, Weather & Commands (set 9) 6 passed

100.0%

IN27 837 OUT5 026

✓

SimpleCmdEchoTimeUnitBFCL/live_simple_157-95-14 (cmd_controller.execute): current time -> 'echo %time%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 619 OUT1 568

✓

SimpleCmdListCDriveUnitBFCL/live_simple_156-95-13 (cmd_controller.execute): list C drive -> 'dir C:\'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 639 OUT695

✓

SimpleCmdStartCameraBFCL/live_simple_160-95-17 (cmd_controller.execute): open camera -> 'start microsoft.windows.camera:'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 614 OUT399

✓

SimpleCmdTaskkillCodeBFCL/live_simple_158-95-15 (cmd_controller.execute): kill VS Code -> 'taskkill /IM code.exe /F'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 614 OUT269

✓

SimpleCmdTimeout10BFCL/live_simple_159-95-16 (cmd_controller.execute): timer 10s -> 'timeout 10'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 634 OUT373

✓

SimpleProvidersChonburiTomorrowBFCL/live_simple_185-110-0 (get_service_providers): ironing, Chonburi, rating>=4.5, tomorrow of 20230929 -> 2023-09-30.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 717 OUT1 722

Tool Calling - Simple: Live Shell, Classification & APIs (set 1) 3 passed

100.0%

IN13 876 OUT5 465

✓

SimpleClassifyGreetingsAndLinksBFCL/live_simple_105-62-0 (classify): clean_hello=['hello'], faq_link_accounts_start=['link my account','connect my accounts'].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 597 OUT611

✓

SimpleCmdDockerVersionBFCL/live_simple_141-94-0 (cmd_controller.execute): command='docker --version'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 576 OUT651

✓

SimpleReqNodesPekAciBFCL/live_simple_128-83-0 (requests.get): nodes list url with ip 192.168.11.33, fabricName='PEK-ACI'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 703 OUT4 203

Tool Calling - Simple: Live Shell, Classification & APIs (set 10) 4 passed

100.0%

IN19 605 OUT3 745

✓

SimpleCmdShutdownBFCL/live_simple_150-95-7 (cmd_controller.execute): shutdown the pc via 'shutdown /s /t 0'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 866 OUT334

✓

SimpleProcessImageMoondreamBFCL/live_simple_122-78-0 (process_data): image.png + long question + model 'vikhyatk/moondream2'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 978 OUT1 436

✓

SimpleReqDataLimit50BFCL/live_simple_135-88-0 (requests.get): data endpoint url, params limit=50.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 897 OUT457

✓

SimpleSumNumbersListBFCL/live_simple_116-72-0 (sum_numbers): numbers_list=[133, 34].1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 864 OUT1 518

Tool Calling - Simple: Live Shell, Classification & APIs (set 11) 4 passed

100.0%

IN20 023 OUT8 822

✓

SimpleCmdWifiDisconnectBFCL/live_simple_151-95-8 (cmd_controller.execute): disconnect wifi via 'netsh wlan disconnect'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 893 OUT390

✓

SimplePipelineFurnitureImageBFCL/live_simple_123-79-0 (pipeline): analyze thewowstyle image with furniture/minimalist question.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 169 OUT3 691

✓

SimpleReqTopologySalesAppBFCL/live_simple_136-89-0 (requests.get): topology url ip 192.168.1.1, filter accountName:AcmeCorp AND applicationName:SalesApp.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 050 OUT1 393

✓

SimpleTemperatureHyderabadBFCL/live_simple_118-74-0 (get_temperature): units=Celsius, location='Hyderabad, India', time omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 911 OUT3 348

Tool Calling - Simple: Live Shell, Classification & APIs (set 12) 4 passed

100.0%

IN21 307 OUT4 796

✓

SimpleCmdDirDAndEchoBFCL/live_simple_152-95-9 (cmd_controller.execute): 'dir D:\ && echo testing.txt'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 273 OUT1 589

✓

SimpleReqAnomaliesSummaryBFCL/live_simple_139-92-0 (requests.get): anomalies summary url ip 192.168.1.1, params fabric='network222'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 394 OUT1 001

✓

SimpleRescheduleAlice8pmBFCL/live_simple_137-90-0 (reschedule): identifier='Alice-One-one-One', 2023-11-01T20:00:00, Europe/London.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 306 OUT348

✓

SimpleWeatherForecastParisBFCL/live_simple_120-76-0 (weather.forecast): Paris, France 2023-04-03..05, Celsius, defaults omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 334 OUT1 858

Tool Calling - Simple: Live Shell, Classification & APIs (set 13) 3 passed

100.0%

IN14 709 OUT3 919

✓

SimpleCmdDelPuttyBFCL/live_simple_153-95-10 (cmd_controller.execute): remove putty shortcut via del with quoted path.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 818 OUT517

✓

SimpleDynamicPricingSaaSBFCL/live_simple_113-69-0 (calculate_dynamic_pricing): geolocation='34.0522, -118.2437', base=100, min=90.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 037 OUT2 136

✓

SimpleRescheduleBob4pmBFCL/live_simple_138-91-0 (reschedule): identifier='Bob-123', 2023-11-01T16:00:00, Europe/London.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 854 OUT1 266

Tool Calling - Simple: Live Shell, Classification & APIs (set 14) 3 passed

100.0%

IN14 094 OUT2 044

✓

SimpleCmdStartYoutubeBFCL/live_simple_154-95-11 (cmd_controller.execute): open youtube rickroll via start command.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 711 OUT406

✓

SimpleGetTicketsCompanyOpenBFCL/live_simple_124-80-0 (get_tickets): customer (company name in prompt), status=open, limit=10.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 696 OUT1 021

✓

SimpleGreetPartnerJeffBFCL/live_simple_140-93-0 (greet_partner): user='Emily', partner='Jeff', relationship='husband'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 687 OUT617

Tool Calling - Simple: Live Shell, Classification & APIs (set 15) 1 passed2 failed

33.3%

IN17 030 OUT7 609

✓

SimpleCalculateTaxNewYorkBFCL/live_simple_104-61-2 (calculate_tax): purchase_amount=250, state='NY', all optionals omitted/default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 832 OUT2 467

×

SimpleCmdDateBFCL/live_simple_155-95-12 (cmd_controller.execute): what is the date via 'date /t' or 'echo %date%'.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 368 OUT4 289

×

SimpleConcatJohnDoeBFCL/live_simple_125-81-0 (concat_strings): strings=['John','Doe'], separator=' '. EXPECTED MODEL MISS: 'combine ... into a full name' requires separator=' ' (John Doe). The 27B tends to omit separator (=> 'JohnDoe'); that is a legit model failure of the measured competence, so the ' ' assertion stays strict.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.separator.AnyOf([" "]) Values: args.separator = null

Failed

IN4 830 OUT853

Tool Calling - Simple: Live Shell, Classification & APIs (set 2) 3 passed

100.0%

IN16 109 OUT6 142

✓

SimpleCmdDirDesktopBFCL/live_simple_142-94-1 (cmd_controller.execute): command='dir Desktop'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 041 OUT951

✓

SimpleRecordBankNineIntentsBFCL/live_simple_106-63-0 (record): classify 20 bank queries into 9 intents (2 buckets empty).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 963 OUT3 359

✓

SimpleReqNodesFabEdBFCL/live_simple_129-83-1 (requests.get): nodes list url with ip 3.321.3232.2, fabricName='fab-ed'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 105 OUT1 832

Tool Calling - Simple: Live Shell, Classification & APIs (set 3) 3 passed1 failed

75.0%

IN35 939 OUT6 122

✓

SimpleCmdDockerPsBFCL/live_simple_143-95-0 (cmd_controller.execute): command='docker ps'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 217 OUT742

×

SimpleRecordTelephoneHoursAccountsBFCL/live_simple_107-64-0 (record): classify 7 queries into telephone-banking / bank-hours / accounts.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN20 120 OUT4 062

✓

SimpleTelemetryGlobalFabricStatusBFCL/live_simple_126-82-0 (telemetry.flowrules.interfaceInfo.get): fabric='Global-Fabric', node=1200, pod=3, ethernet, status.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 299 OUT510

✓

SimpleUpdateProfileEmailAgeBFCL/live_simple_114-70-0 (update_user_profile): user_id=12345, email+age set, bio omitted, notify default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 303 OUT808

Tool Calling - Simple: Live Shell, Classification & APIs (set 4) 3 passed1 failed

75.0%

IN20 858 OUT5 354

✓

SimpleCmdKillFirefoxBFCL/live_simple_144-95-1 (cmd_controller.execute): close firefox via taskkill.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 081 OUT352

×

SimpleRecordMoneyTransactionsCreditBFCL/live_simple_108-65-0 (record): classify 10 queries into money-movement / transactions / credit-limit / link-accounts. EXPECTED MODEL MISS: the 27B classifies 'show me all outgoing wire transfers' as a transaction-view (get_transactions_start) rather than money_movement_start per GT, so the get_transactions count comes out as 5 instead of 4. GT bucketing kept strict; this is a legit model classification failure.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: args.money_movement_start.Count == 1 Values: args.money_movement_start.Count = 0

Failed

IN5 549 OUT3 172

✓

SimpleReverseSayHiBFCL/live_simple_117-73-0 (reverse_input): input_value='say hi'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 069 OUT1 079

✓

SimpleTelemetryTestDeStatusBFCL/live_simple_127-82-1 (telemetry.flowrules.interfaceInfo.get): fabric='test-de', node=5, pod=1, ethernet, status.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 159 OUT751

Tool Calling - Simple: Live Shell, Classification & APIs (set 5) 3 passed1 failed

75.0%

IN20 854 OUT5 854

✓

SimpleCmdDirCDriveBFCL/live_simple_145-95-2 (cmd_controller.execute): list C drive => 'dir C:\'. EXPECTED MODEL MISS: for the terse/typo'd 'LIST C DRIIVE' the 27B currently returns 'dir d:' — the wrong drive. The accepted-command list is broadened with the no-backslash 'dir c:' variant for fairness, but a wrong drive letter is a legit model failure.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 105 OUT1 306

✓

SimplePrimeMinistersThreeBFCL/live_simple_119-75-0 (GetPrimeMinisters): countries=[Australia, Canada, India].1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 149 OUT991

✓

SimpleRecordAutoWithdrawIncomeBFCL/live_simple_109-66-0 (record): classify 5 loan/withdraw queries; outofscope empty.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 338 OUT1 463

×

SimpleReqInterfaceStatsNetwork1BFCL/live_simple_130-84-0 (requests.get): interfaceInfo stats, ip 192.120.45.67, Network1/node 12/pod 10/ethernet/statistics.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.params.infoType?.ToLower()).AnyOf(["statistics"])

Failed

IN5 262 OUT2 094

Tool Calling - Simple: Live Shell, Classification & APIs (set 6) 4 passed

100.0%

IN21 521 OUT3 844

✓

SimpleCmdStartHuggingfaceBFCL/live_simple_146-95-3 (cmd_controller.execute): open huggingface via start command.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 292 OUT442

✓

SimpleGenerateSalaryChartBFCL/live_simple_121-77-0 (generate_chart): 5 job labels and salaries in order.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 446 OUT535

✓

SimpleRecordAtmLocationOnlyBFCL/live_simple_110-67-0 (record): single query 'Need ATM location' -> atm_finder_start only.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 291 OUT635

✓

SimpleReqInterfaceSpeedNetwork1BFCL/live_simple_131-84-1 (requests.get): interfaceInfo Speed, ip 192.120.45.67, Network1/node 12/pod 10/ethernet/Speed.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 492 OUT2 232

Tool Calling - Simple: Live Shell, Classification & APIs (set 7) 3 passed

100.0%

IN14 909 OUT2 713

✓

SimpleCmdKillTimerBFCL/live_simple_147-95-4 (cmd_controller.execute): remove timer.exe via taskkill.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 865 OUT595

✓

SimpleRecordBalanceAndAtmBFCL/live_simple_111-67-1 (record): 10 queries -> 8 balance + 2 atm; other buckets empty.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 172 OUT1 068

✓

SimpleReqDeviceConnectorVersionsBFCL/live_simple_132-85-0 (requests.get): device connector versions JSON url, no params.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 872 OUT1 050

Tool Calling - Simple: Live Shell, Classification & APIs (set 8) 2 passed1 failed

66.7%

IN14 863 OUT3 491

×

SimpleCmdWhatFileInCBFCL/live_simple_148-95-5 (cmd_controller.execute): 'what is the file in c' => 'dir C:\'. EXPECTED MODEL MISS: the 27B mis-selects the 'record' classifier (dropping the request into its 'outofscope' bucket) instead of cmd_controller_execute. record is a legitimate distractor here (not a looping universal collider), so this stays as a real tool-selection failure — no isolation needed.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: call.ToolName == expectedTool Values: call.ToolName = "record" expectedTool = "cmd_controller_execute"

Failed

IN4 943 OUT1 824

✓

SimpleRecordWeatherOutOfScopeBFCL/live_simple_112-68-0 (record): single query 'what is the weather like' -> outofscope only.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 958 OUT393

✓

SimpleReqRecommendationsDacBFCL/live_simple_133-86-0 (requests.get): recommendations url, params advisoryId='dac'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 962 OUT1 274

Tool Calling - Simple: Live Shell, Classification & APIs (set 9) 3 passed

100.0%

IN13 871 OUT2 943

✓

SimpleCalculateSum133And34BFCL/live_simple_115-71-0 (calculate_sum): number1=133, number2=34.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 607 OUT683

✓

SimpleCmdEchoHiBFCL/live_simple_149-95-6 (cmd_controller.execute): say hi via echo command.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 581 OUT1 051

✓

SimpleReqInsightsGroupsBFCL/live_simple_134-87-0 (requests.get): insights groups url, params limit=20, sort='desc'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 683 OUT1 209

Tool Calling - Simple: Math, Physics & Geometry (set 1) 12 passed

100.0%

IN88 627 OUT16 742

✓

SimpleAlgebraQuadraticRootsBFCL/simple_python_3 (algebra.quadratic_roots): a=1, b=-3, c=2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 395 OUT1 218

✓

SimpleAreaUnderCurveXSquaredBFCL/simple_python_13 (calculate_area_under_curve): function y=x^2, interval [1,3], method omitted/trapezoidal.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 388 OUT2 021

✓

SimpleCircleCircumferenceInchesBFCL/simple_python_7 (calculate_circumference): radius=4, unit inches/in.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 376 OUT1 026

✓

SimpleDerivativeCubicPolyBFCL/simple_python_14 (calculate_derivative): function 3x^2+2x-1, x_value omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 383 OUT1 626

✓

SimpleElectricFieldFromChargeBFCL/simple_python_39 (calculate_electric_field): charge=2, distance=3, permitivity omitted/8.854e-12.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 384 OUT1 347

✓

SimpleFinalVelocityFromDistanceBFCL/simple_python_30 (kinematics.final_velocity_from_distance): acceleration=4, distance=300, initial_velocity omitted/0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 420 OUT1 887

✓

SimpleFinalVelocityFromHeightBFCL/simple_python_25 (calculate_final_velocity): height=150, initial_velocity omitted/0, gravity omitted/9.81.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 392 OUT1 008

✓

SimpleGeometryAreaCircleBFCL/simple_python_8 (geometry.area_circle): radius=10, units omitted or 'meters'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 365 OUT1 135

✓

SimpleGetPrimeFactors450BFCL/simple_python_17 (get_prime_factors): number=450, formatted omitted or true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 355 OUT1 075

✓

SimpleMagneticFieldLoopBFCL/simple_python_40 (calculate_magnetic_field): current=5, radius=4, permeability omitted/12.57e10.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 396 OUT1 824

✓

SimpleMathGcd40And50BFCL/simple_python_19 (math.gcd): num1=40, num2=50.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 379 OUT1 784

✓

SimpleTriangleAreaWithUnitBFCL/simple_python_0 (calculate_triangle_area): base=10, height=5, unit omitted or 'units'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 394 OUT791

Tool Calling - Simple: Math, Physics & Geometry (set 2) 12 passed

100.0%

IN88 315 OUT16 069

✓

SimpleAverageVelocityBFCL/simple_python_26 (calculate_velocity): distance=50, duration=2, unit omitted or 'km/h'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 359 OUT616

✓

SimpleCalculusDerivativeAtValueBFCL/simple_python_16 (calculus.derivative): function 2x^2, value=1, function_variable omitted or 'x'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 350 OUT1 363

✓

SimpleElectricFieldStrengthBFCL/simple_python_44 (calculate_electric_field_strength): charge=0.01, distance=4, medium omitted or 'vacuum'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 360 OUT2 333

✓

SimpleFinalVelocityConstAccelBFCL/simple_python_31 (calculate_final_velocity): initial_velocity=0, acceleration=9.8, time=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 397 OUT935

✓

SimpleGeometryCalculateAreaCircleBFCL/simple_python_9 (geometry.calculate_area_circle): radius=5, unit omitted or 'units'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 343 OUT1 261

✓

SimpleGeometryCircumferenceBFCL/simple_python_12 (geometry.circumference): radius=3, units omitted or 'cm'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 327 OUT1 178

✓

SimpleIntegrateCubeSimpsonBFCL/simple_python_15 (integrate): function x^3, start_x=-2, end_x=3, method simpson.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 392 OUT1 340

✓

SimpleMagneticFieldStrengthWireBFCL/simple_python_43 (calculate_magnetic_field_strength): current=20, distance=10, permeability omitted/12.57e-7.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 367 OUT2 348

✓

SimpleMathHcfBFCL/simple_python_20 (math.hcf): number1=36, number2=24.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 345 OUT728

✓

SimpleNumberAnalysisPrimeFactorsBFCL/simple_python_18 (number_analysis.prime_factors): number=123456.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 341 OUT1 194

✓

SimpleRightTriangleAreaCmBFCL/simple_python_10 (calculate_area): base=6, height=10, unit omitted or 'cm'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 376 OUT872

✓

SimpleSolveQuadraticEquation265BFCL/simple_python_4 (solve_quadratic_equation): a=2, b=6, c=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 358 OUT1 901

Tool Calling - Simple: Math, Physics & Geometry (set 3) 9 passed

100.0%

IN60 210 OUT11 039

✓

SimpleBoilingMeltingPointsBFCL/simple_python_47 (get_boiling_melting_points): substance='water', sea_level=5000.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 684 OUT2 058

✓

SimpleDisplacementBFCL/simple_python_28 (calculate_displacement): initial_velocity=10, time=5, acceleration=9.8.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 702 OUT1 967

✓

SimpleElectrostaticPotentialBFCL/simple_python_38 (calculate_electrostatic_potential): charge1=1e-9, charge2=2e-9, distance=0.05, constant omitted/8.99e9.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 716 OUT1 663

✓

SimpleFinalSpeedFromHeightBFCL/simple_python_32 (calculate_final_speed): initial_velocity=0, height=100, gravity omitted or 9.8.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 672 OUT810

✓

SimpleNumberTheoryGcdBFCL/simple_python_21 (number_theory.gcd): number1=36, number2=48.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 685 OUT922

✓

SimplePrimeFactorizeDictionaryBFCL/simple_python_23 (prime_factorize): number=60, return_type='dictionary'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 676 OUT854

✓

SimpleRouteEstimateTimeStopsBFCL/simple_python_37 (route.estimate_time): SF -> LA, stops {Santa Barbara, Monterey} in either order.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 694 OUT879

✓

SimpleSolveQuadraticAllRootsBFCL/simple_python_5 (solve_quadratic): a=3, b=-11, c=-4, root_type='all' (user asked for all roots).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 705 OUT1 188

✓

SimpleTriangleAreaNoUnitBFCL/simple_python_11 (calculate_triangle_area): base=10, height=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 676 OUT698

Tool Calling - Simple: Math, Physics & Geometry (set 4) 9 passed

100.0%

IN59 747 OUT12 916

✓

SimpleCalculateDensityBFCL/simple_python_48 (calculate_density): mass=45, volume=15, unit omitted or 'kg/m^3'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 640 OUT789

✓

SimpleElectromagneticForceBFCL/simple_python_41 (electromagnetic_force): charge1=5, charge2=7, distance=3, medium_permittivity omitted/8.854e-12.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 642 OUT1 792

✓

SimpleFinalSpeedFreeFallTimeBFCL/simple_python_29 (calculate_final_speed): initial_speed omitted/0, time=5, gravity omitted/-9.81.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 636 OUT1 351

✓

SimpleMathFactorial5BFCL/simple_python_1 (math.factorial): number=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 600 OUT1 158

✓

SimpleMathGcd12And15BFCL/simple_python_22 (math.gcd): num1=12, num2=15.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 633 OUT561

✓

SimpleShortestDrivingDistanceBFCL/simple_python_36 (get_shortest_driving_distance): origin=New York City, destination=Washington D.C., unit omitted/km.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 625 OUT1 704

✓

SimpleSolveQuadraticRoots253BFCL/simple_python_6 (solve_quadratic): a=2, b=5, c=3.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 638 OUT1 427

✓

SimpleThermoEnergyVaporizationBFCL/simple_python_45 (thermo.calculate_energy): mass=100, phase_transition=vaporization, substance omitted/water.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 670 OUT1 644

✓

SimpleTravelItineraryTokyoBFCL/simple_python_34 (travel_itinerary_generator): destination=Tokyo, days=7, daily_budget=100, exploration_type=nature.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 663 OUT2 490

Tool Calling - Simple: Math, Physics & Geometry (set 5) 8 passed

100.0%

IN51 550 OUT11 020

✓

SimpleAbsolutePressureBFCL/simple_python_49 (calc_absolute_pressure): atm_pressure=1, gauge_pressure=2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 437 OUT1 703

✓

SimpleFinalTemperatureMixBFCL/simple_python_46 (calculate_final_temperature): mass1=20, temperature1=30, mass2=15, temperature2=60, shc omitted/4.2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 475 OUT2 837

✓

SimpleFinalVelocityAccelTimeBFCL/simple_python_27 (final_velocity): initial_velocity=10, acceleration=2, time=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 469 OUT912

✓

SimpleGetDirectionsFastestBFCL/simple_python_33 (get_directions): start=Sydney, end=Melbourne, route_type omitted or 'fastest'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 422 OUT1 022

✓

SimpleMathGcd12And18BFCL/simple_python_24 (math.gcd): num1=12, num2=18.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 423 OUT699

✓

SimpleMathHypotenuseBFCL/simple_python_2 (math.hypot): x=4, y=5, z omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 433 OUT1 219

✓

SimpleResonantFrequencyLCBFCL/simple_python_42 (calculate_resonant_frequency): inductance=0.05 (50mH), capacitance=0.0001 (100uF), round_off omitted/2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 460 OUT1 415

✓

SimpleVeganRestaurantFindBFCL/simple_python_35 (vegan_restaurant.find_nearby): location='New York, NY', operating_hours=23 (11 PM).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 431 OUT1 213

Tool Calling - Simple: Science, Data & Biology (set 1) 13 passed

100.0%

IN107 019 OUT20 864

✓

SimpleArraySortBFCL/simple_python_87 (array_sort): list=[5,3,4,1,2], order=ascending.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 213 OUT1 279

✓

SimpleAverageTemperatureBFCL/simple_python_78 (average_temperature): location=Austin, days=3, temp_unit=Celsius.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 197 OUT765

✓

SimpleBacteriaEvolutionRateBFCL/simple_python_74 (calculate_bacteria_evolution_rate): start=5000, dup_freq=1, duration=6, gen_time omitted or 20.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 230 OUT2 341

✓

SimpleBmiKgCmBFCL/simple_python_84 (calculate_bmi): weight=85, height=180, unit omitted or 'metric'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 225 OUT2 460

✓

SimpleCreateHistogramBFCL/simple_python_79 (create_histogram): data=[85,90,88,92,86,89,91], bins=5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 273 OUT693

✓

SimpleCreatureFitnessBFCL/simple_python_72 (calculate_fitness): trait_values=[0.8,0.7], trait_contributions=[0.4,0.6].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 276 OUT1 951

✓

SimpleDbFetchStudentsBFCL/simple_python_89 (db_fetch_records): database=StudentDB, table=students, conditions={department:Science, school:Bluebird High School}, fetch_limit omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 227 OUT1 130

✓

SimpleEntropyChangeSubstanceBFCL/simple_python_50 (entropy_change.calculate): substance=ice, mass=1, init=0, final=100, pressure omitted or 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 265 OUT4 343

✓

SimpleGenerateDnaSequenceBFCL/simple_python_71 (generate_DNA_sequence): length=100, preferences=[G,C] in either order.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 216 OUT952

✓

SimpleGpsCoordinateDistanceBFCL/simple_python_83 (calculate_distance): coord1=[33.4484,-112.074], coord2=[34.0522,-118.2437], unit=miles.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 305 OUT1 729

✓

SimpleImdbMoviesByActorBFCL/simple_python_92 (imdb.find_movies_by_actor): actor='Leonardo DiCaprio', year=2010, category omitted or 'all'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 210 OUT744

✓

SimplePopulationProjectionBFCL/simple_python_73 (population_projections): country=United States/USA, years=20, growth_rate omitted or 1.2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 193 OUT1 719

✓

SimpleRestaurantFindNearbyBFCL/simple_python_77 (restaurant.find_nearby): location='Los Angeles, CA', dietary_preference=[Vegan].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 189 OUT758

Tool Calling - Simple: Science, Data & Biology (set 2) 13 passed

100.0%

IN106 624 OUT13 702

✓

SimpleBmiMetersBFCL/simple_python_88 (calculate_BMI): weight_kg=70, height_m=1.75.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN8 202 OUT1 245

✓

SimpleClockAngleBFCL/simple_python_98 (calculate_clock_angle): hours=6, minutes=30, round_to omitted or 2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 186 OUT996

✓

SimpleDatabaseQueryConditionsBFCL/simple_python_96 (database.query): table=user, conditions=[{age,>,25},{job,=,engineer}].1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 225 OUT924

✓

SimpleElephantPopulationBFCL/simple_python_75 (elephant_population_estimate): current=35000, growth_rate=0.015, years=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN8 226 OUT1 823

✓

SimpleEntropyChangeIsothermalBFCL/simple_python_51 (calculate_entropy_change): initial_temp=300, final_temp=400, heat_capacity=5, isothermal omitted or true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 234 OUT1 273

✓

SimpleEvolutionPredictionDarwinBFCL/simple_python_76 (prediction.evolution): species=Homo Sapiens, years=50, model=Darwin.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 200 OUT846

✓

SimpleFindRestaurantsThaiBFCL/simple_python_80 (find_restaurants): location=Manhattan variants, food_type=Thai, number=5, dietary_requirements=[vegan].1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 205 OUT1 647

✓

SimpleGeoDistanceCitiesBFCL/simple_python_85 (geo_distance.calculate): start='Boston, MA', end='Washington, D.C.', units omitted or 'miles'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 197 OUT1 206

✓

SimpleMathFactorialBFCL/simple_python_97 (math.factorial): number=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN8 149 OUT548

✓

SimplePlotSineWaveBFCL/simple_python_99 (plot_sine_wave): start=0.0, end=6.2832, frequency=5, amplitude omitted or 1, phase_shift omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 197 OUT544

✓

SimpleTheaterMovieReleasesBFCL/simple_python_93 (get_theater_movie_releases): location=LA, timeframe=7, format=IMAX.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 191 OUT848

✓

SimpleTriangleAreaMetersBFCL/simple_python_95 (calc_area_triangle): base=5, height=3.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN8 175 OUT764

✓

SimpleUpdateUserInfoBFCL/simple_python_94 (update_user_info): user_id=43523, update_info={name:John Doe, email:johndoe@email.com}, database omitted or CustomerInfo. NOTE: single-call contract kept intentionally; the model very occasionally emits a spurious second call (args are correct ~3/3 sampled runs), a model-reliability signal rather than a test defect.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 237 OUT1 038

Tool Calling - Simple: Science, Data & Biology (set 3) 11 passed1 failed

91.7%

IN95 427 OUT15 226

✓

SimpleCalculateAverageBFCL/simple_python_82 (calculate_average): numbers=[12,15,18,20,21,26,30].1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 660 OUT879

✓

SimpleCellDensityBFCL/simple_python_57 (calculate_cell_density): optical_density=0.6, dilution=5, calibration_factor omitted or 1e9.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 632 OUT1 549

✓

SimpleCellInfoHumanBFCL/simple_python_55 (biology.get_cell_info): cell_type=human, detailed=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 596 OUT1 548

×

SimpleCellProteinsMembraneBFCL/simple_python_56 (cellbio.get_proteins): cell_compartment=plasma membrane, include_description omitted/true/false.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 562 OUT3 139

✓

SimpleCityDistanceTrainBFCL/simple_python_86 (city_distance.find_shortest): start=New York, end=Los Angeles, transportation=train, allow_transfer=true.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 640 OUT1 329

✓

SimpleEmployeeFetchDataBFCL/simple_python_90 (employee.fetch_data): company='ABC Ltd.', employee_id=345, data_field=[Personal Info, Job History].1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 641 OUT424

✓

SimpleFetchDnaSequenceBFCL/simple_python_53 (fetch_DNA_sequence): DNA_id=DNA123, format omitted or 'fasta', upstream omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 599 OUT663

✓

SimpleGetRestaurantSushiBFCL/simple_python_91 (get_restaurant): cuisine=sushi, location=Boston, condition=open on Sundays.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 614 OUT1 018

✓

SimpleHeatCapacityAirBFCL/simple_python_52 (calc_heat_capacity): temp=298, volume=10, gas omitted or 'air'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 643 OUT1 930

✓

SimpleMolecularWeightBFCL/simple_python_59 (calculate_molecular_weight): compound=C6H12O6, to_unit=grams/mole or g/mol.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 626 OUT1 102

✓

SimpleOrganelleFunctionLookupBFCL/simple_python_58 (cell_biology.function_lookup): molecule=ATP synthase, organelle=mitochondria, specific_function=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 615 OUT915

✓

SimpleProteinSequenceBFCL/simple_python_54 (get_protein_sequence): gene=BRCA1, species omitted or 'Homo sapiens'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 599 OUT730

Tool Calling - Simple: Science, Data & Biology (set 4) 12 passed

100.0%

IN94 138 OUT17 441

✓

SimpleAnalyzeDnaSequenceBFCL/simple_python_62 (analyze_dna_sequence): sequence + reference_sequence given, mutation_type omitted or 'substitution'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 900 OUT1 403

✓

SimpleDiabetesPredictionBFCL/simple_python_61 (diabetes_prediction): weight=150, height=70 (5ft10in), activity_level=lightly active.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 861 OUT1 468

✓

SimpleFastestRouteAvoidTollsBFCL/simple_python_81 (map_routing.fastest_route): start=San Francisco/SF, end=Los Angeles/LA, avoid_tolls=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 830 OUT592

✓

SimpleForestGrowthForecastBFCL/simple_python_68 (forest_growth_forecast): location=Yellowstone National Park, years=5, include_human_impact=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 837 OUT2 126

✓

SimpleGeneticSimilarityBFCL/simple_python_63 (genetics.calculate_similarity): species1=Human, species2=Chimp/Chimpanzee, format omitted or 'percentage'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 830 OUT1 490

✓

SimpleGenotypeFrequencyBFCL/simple_python_64 (calculate_genotype_frequency): allele_frequency=0.3, genotype=AA.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 839 OUT957

✓

SimpleIdentifyBirdBFCL/simple_python_67 (identify_bird): color=green, habitat=forest, size=small.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 813 OUT1 210

✓

SimpleMutationTypeFindBFCL/simple_python_60 (mutation_type.find): snp_id=rs6034464, species omitted or 'Homo sapiens'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 843 OUT2 241

✓

SimplePopulationDensityBFCL/simple_python_65 (calculate_density): country=Brazil, year='2022', population=213000000, land_area=8500000.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 886 OUT1 676

✓

SimplePrecipitationStatsBFCL/simple_python_66 (ecology_data.precipitation_stats): location=Amazon rainforest, time_frame=six_months.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 828 OUT987

✓

SimpleTurtlePopulationBFCL/simple_python_69 (ecology.get_turtle_population): location=Mississippi river, year=2020, species=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 836 OUT1 348

✓

SimpleVehicleEmissionBFCL/simple_python_70 (calculate_vehicle_emission): vehicle_type=gas, miles_driven=1500, emission_factor omitted or 355.48.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 835 OUT1 943

Tool Calling - Simple: Sports & Games (set 1) 11 passed1 failed

91.7%

IN95 911 OUT15 977

✓

SimpleBattingAverageComputeBFCL/simple_python_302 (calculate_batting_average): hits=180, at_bats=600, decimal_places=3 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 667 OUT902

✓

SimpleBoardGameInfoTicketBFCL/simple_python_328 (boardgame.get_info): Ticket to Ride, {rating, player count} either order, language omitted/English.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 628 OUT1 411

✓

SimpleFindCardInDeckBFCL/simple_python_335 (find_card_in_deck): rank Queen, suit Hearts, deck omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 629 OUT541

✓

SimpleGameListByYearBFCL/simple_python_342 (game_list.get_games): year 2019, multiplayer true, ESRB Everyone.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 650 OUT891

✓

SimpleGameWinnerByDateBFCL/simple_python_307 (game_result.get_winner): Lakers/Clippers either order, date 2021-01-28, venue omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 656 OUT992

✓

SimpleHighestGameScoreBFCL/simple_python_349 (game_score.highest): Overwatch, PC, region Global or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 641 OUT1 652

✓

SimpleMatchScheduleBFCL/simple_python_308 (sports.match_schedule): Man United, 5 matches, English Premier League or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 646 OUT1 827

✓

SimpleNoteDurationBFCL/simple_python_300 (music.calculate_note_duration): 440/880 Hz, tempo omitted or 120.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 648 OUT1 367

✓

SimplePokerGameWinnerMultiBFCL/simple_python_337 (poker_game_winner): 4 players + per-player hands dict, type omitted/Texas Holdem.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 839 OUT1 821

✓

SimpleSoccerPlayerStatsBFCL/simple_python_303 (soccer_stat.get_player_stats): Cristiano Ronaldo, 2019-2020, league omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 646 OUT913

×

SimpleSportRankingPlayerBFCL/simple_python_316 (get_sport_ranking): Tennis, Serena Williams, gender omitted/all/female.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 614 OUT1 863

✓

SimpleTeamRankRegularBFCL/simple_python_317 (get_team_rank): LA Lakers, NBA, 2021, type regular.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 647 OUT1 797

Tool Calling - Simple: Sports & Games (set 2) 10 passed1 failed

90.9%

IN86 088 OUT21 114

✓

SimpleBoardGameInfoCatanBFCL/simple_python_330 (board_game_info): Catan, {average_review_rating, age_range}.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 123 OUT1 870

✓

SimpleCricketBattingScoreBFCL/simple_python_306 (average_batting_score): Virat Kohli, 10 matches, T20 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 125 OUT2 332

✓

SimpleFifaTeamRankingBFCL/simple_python_318 (get_team_ranking): Germany, 2021, gender men or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 132 OUT736

✓

SimpleMatchResultsTwoTeamsBFCL/simple_python_325 (sports.match_results): Chicago Bulls vs Los Angeles Lakers, season omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 110 OUT2 866

✓

SimplePokerDetermineWinnerTwoBFCL/simple_python_340 (card_games.poker_determine_winner): John vs Mike with 5-card hands (suit symbols).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 286 OUT1 746

✓

SimpleShuffleAndDrawBFCL/simple_python_336 (cards.shuffle_and_draw): num_cards 3.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 104 OUT537

✓

SimpleSoccerPerformanceBFCL/simple_python_305 (sports_stats.get_performance): Messi, La Liga, 2020-2021, {goals scored, assists made}.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 160 OUT1 321

✓

SimpleTeamScheduleGamesBFCL/simple_python_327 (sports_team.get_schedule): Man United, 6 games, Premier League, location omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 128 OUT2 742

✓

SimpleThirdChordBFCL/simple_python_301 (get_third_chord): key C, type major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 102 OUT1 682

×

SimpleTopPlayerBySportBFCL/simple_python_323 (sports_ranking.get_top_player): tennis, gender women.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 698 OUT4 805

✓

SimpleVideoGamePlayerStatsBFCL/simple_python_343 (game_stats.fetch_player_statistics): Zelda, user Sam, platform Switch.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 120 OUT477

Tool Calling - Simple: Sports & Games (set 3) 9 passed

100.0%

IN58 082 OUT12 501

✓

SimpleCardDrawProbabilityBFCL/simple_python_338 (card_game_probability.calculate): total 52, desired 13 (hearts), cards_drawn 1 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 464 OUT951

✓

SimpleCelebrityNetWorthBFCL/simple_python_313 (celebrity_net_worth.get): Messi, currency EUR/euro.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 448 OUT1 708

✓

SimpleDefenseRankingBFCL/simple_python_315 (get_defense_ranking): season 2021, top 1 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 447 OUT1 170

✓

SimpleFindAthleteBFCL/simple_python_311 (sports_db.find_athlete): Lebron James, sport Basketball, team omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 440 OUT1 127

✓

SimpleGameItemStatsBFCL/simple_python_344 (get_game_item_stats): BotW, Guardian Sword+, stat power/power rating.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 484 OUT1 201

✓

SimpleMonopolyOddsBFCL/simple_python_329 (monopoly_odds_calculator): number 7, dice_number 2, dice_faces 6 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 461 OUT2 031

✓

SimpleNflPlayerRecordBFCL/simple_python_309 (nfl_data.player_record): Tom Brady, 2020, team omitted or Tampa Bay Buccaneers.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 454 OUT1 001

✓

SimpleSportsRankingSeasonIntBFCL/simple_python_319 (sports_ranking): Man United, Premier League, season omitted or 2023.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 438 OUT1 012

✓

SimpleTeamLatestScoreBFCL/simple_python_324 (team_score.get_latest): Los Angeles Lakers, include_opponent true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 446 OUT2 300

Tool Calling - Simple: Sports & Games (set 4) 7 passed

100.0%

IN41 849 OUT10 823

✓

SimpleCareerStatsBFCL/simple_python_310 (get_career_stats): LeBron James, team omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 954 OUT2 067

✓

SimpleCelebrityAchievementsBFCL/simple_python_314 (sports_celebrity.get_major_achievements): Messi, sports Football/Soccer or omitted, team all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 972 OUT1 712

✓

SimpleChessTopPlayersBFCL/simple_python_331 (board_game.chess.get_top_players): New York, min rating 2300, number_of_players 10 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 979 OUT1 377

✓

SimpleDeckOddsSuitBFCL/simple_python_341 (deck_of_cards.odds): suit hearts, deck_type without_joker or normal.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 971 OUT1 222

✓

SimpleGameValuationBFCL/simple_python_345 (game_valuation): Super Mario Bros., 1985, condition Like New/New.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 991 OUT1 783

✓

SimpleTeamPositionDetailedBFCL/simple_python_320 (sports_ranking.get_team_position): Golden State Warriors, 2022-2023, detailed true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 007 OUT1 136

✓

SimpleTeamScoreWithStatsBFCL/simple_python_326 (get_team_score): Los Angeles Lakers, NBA, include_player_stats any.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 975 OUT1 526

Tool Calling - Simple: Sports & Games (set 5) 7 passed

100.0%

IN40 513 OUT12 731

✓

SimpleChessRatingBFCL/simple_python_332 (chess.rating): Magnus Carlsen, variant classical or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 772 OUT1 486

✓

SimpleCollectablesInSeasonBFCL/simple_python_346 (get_collectables_in_season): Animal Crossing: New Horizons, Spring, item_type all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 795 OUT1 264

✓

SimpleDetailedWeatherForecastBFCL/simple_python_333 (detailed_weather_forecast): London UK, 3 days, {high/low temp, humidity, precipitation}.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 818 OUT2 301

✓

SimpleFullHouseProbabilityBFCL/simple_python_339 (poker_probability.full_house): defaults deck_size 52, hand_size 5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 773 OUT1 709

✓

SimplePlayerStatisticYearBFCL/simple_python_312 (player_statistic): Ronaldo, year 2021, team omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 779 OUT2 104

✓

SimpleSoccerLastMatchBFCL/simple_python_347 (soccer.get_last_match): Liverpool F.C., include_stats true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 785 OUT3 147

✓

SimpleSportsRankingSeasonStrBFCL/simple_python_321 (sports_ranking): Barcelona, La Liga, 2021.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 791 OUT720

Tool Calling - Simple: Sports & Games (set 6) 4 passed

100.0%

IN20 199 OUT4 899

✓

SimpleBlackjackWinnerBFCL/simple_python_334 (blackjack.check_winner): player [A,10], dealer [10,9], ace_value 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 080 OUT1 486

✓

SimpleCreatePlayerProfileBFCL/simple_python_348 (create_player_profile): StarPlayer, class Mage, starting_level 5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 057 OUT299

✓

SimpleCurrentTeamRankingBFCL/simple_python_322 (sports_ranking.get_current): Liverpool, Premier League, season omitted or 2023-2024.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 021 OUT1 358

✓

SimplePlayerLastGameStatsBFCL/simple_python_304 (player_stats.getLastGame): LeBron James, Lakers, metrics {Points, Rebounds}.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 041 OUT1 756

Tool Calling - Simple: Stats, Finance & ML (set 1) 8 passed

100.0%

IN51 703 OUT13 126

✓

SimpleCardProbabilityBFCL/simple_python_112 (calculate_probability): total_outcomes=52, favorable_outcomes=4 (king from deck), round_to omitted or 2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 411 OUT1 013

✓

SimpleCompoundInterestNBFCL/simple_python_131 (calculate_compound_interest): principal=10000, rate=0.05, time=10, n=4.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 509 OUT1 957

✓

SimpleInvestmentProfitBFCL/simple_python_134 (investment.predictProfit): investment_amount=5000, annual_return=0.07, years=5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 454 OUT1 484

✓

SimpleLightTravelTimeBFCL/simple_python_100 (light_travel_time): distance_in_light_years=4, speed_of_light omitted or default.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 417 OUT558

✓

SimpleQuarterlyDividendBFCL/simple_python_128 (finance.calculate_quarterly_dividend_per_share): total_payout=50000000, outstanding_shares=100000000.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 465 OUT2 638

✓

SimpleRandomForestTrainBFCL/simple_python_109 (random_forest.train): n_estimators=100, max_depth=5, data='my_data'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 443 OUT366

✓

SimpleStockPriceByDateBFCL/simple_python_142 (get_stock_price): company_name Amazon/AMZN, date=2022-03-11, exchange NASDAQ or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 445 OUT2 292

✓

SimpleTTestStatsBFCL/simple_python_118 (stats.t_test): array_1=[10,15,12,14,11], array_2=[18,16,17,20,22], alpha=0.05.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 559 OUT2 818

Tool Calling - Simple: Stats, Finance & ML (set 2) 8 passed

100.0%

IN53 920 OUT11 395

✓

SimpleCalculateSpeedBFCL/simple_python_101 (calculate_speed): distance=450, time=20, to_unit=km/h.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 717 OUT1 139

✓

SimpleCardProbabilitySingleBFCL/simple_python_116 (probabilities.calculate_single): total_outcomes=52, event_outcomes=4 (king), round omitted or 2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 705 OUT720

✓

SimpleCompoundInterestFreqBFCL/simple_python_136 (compound_interest): principal=10000, annual_rate=5.0, compounding_freq=monthly, time_in_years=5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 742 OUT1 416

✓

SimpleDiscountedCashFlowBFCL/simple_python_129 (calculate_discounted_cash_flow): coupon_payment=100, period=5, discount_rate=0.04, face_value omitted or 1000.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 737 OUT1 738

✓

SimpleHousePriceBedroomsBFCL/simple_python_110 (predict_house_price): bedrooms=3, bathrooms=2, area=1800, location San Francisco.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 738 OUT1 887

✓

SimpleStockPriceDaysBFCL/simple_python_143 (get_stock_price): company=AAPL, days=5, exchange NASDAQ or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 698 OUT1 472

✓

SimpleStockReturnProjectedBFCL/simple_python_137 (calculate_stock_return): investment_amount=5000, annual_growth_rate=0.06, holding_period=5, dividends omitted or false.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 750 OUT1 627

✓

SimpleTTestIndBFCL/simple_python_119 (hypothesis_testing.ttest_ind): sample1=[22,33,42,12,34], sample2=[23,45,44,14,38], significance_level=0.05.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 833 OUT1 396

Tool Calling - Simple: Stats, Finance & ML (set 3) 8 passed

100.0%

IN52 218 OUT14 883

✓

SimpleCompoundedInterestAnnuallyBFCL/simple_python_145 (calculate_compounded_interest): principal=5000, interest_rate=0.05, period=10, compounding_frequency omitted or Annually.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 523 OUT1 314

✓

SimpleDistanceEarthMoonBFCL/simple_python_102 (calculate_distance): body1=Earth, body2=Moon, unit mi/miles/mile.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 479 OUT1 363

✓

SimpleHeartRatioProbabilityBFCL/simple_python_117 (probability_of_event): success_outcomes=13 (hearts), total_outcomes=52, format_as_ratio=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 511 OUT1 233

✓

SimpleHousePriceRoomsBFCL/simple_python_125 (predict_house_price): area=2500, rooms=5, year=1990, location San Francisco/SF.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 538 OUT897

✓

SimpleMutualFundReturnBFCL/simple_python_139 (estimate_mutual_fund_return): yearly_yield=5.0, investment_amount=2000, years=3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 539 OUT1 900

✓

SimpleReturnOnEquityBFCL/simple_python_132 (calculate_return_on_equity): net_income=2000000, shareholder_equity=10000000, dividends_paid=200000.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 581 OUT4 803

✓

SimpleStockPriceCurrentMultiBFCL/simple_python_149 (get_stock_price): company_names=[Apple, Microsoft] (or tickers AAPL, MSFT).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 469 OUT1 433

✓

SimpleTwoSampleTTestRunBFCL/simple_python_120 (run_two_sample_ttest): group1=[3,4,5,6,4], group2=[7,8,9,8,7], equal_variance=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 578 OUT1 940

Tool Calling - Simple: Stats, Finance & ML (set 4) 7 passed1 failed

87.5%

IN52 854 OUT13 241

✓

SimpleBinomialProbDistBFCL/simple_python_114 (prob_dist.binomial): trials=10, successes=5, p omitted or 0.5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 564 OUT2 326

×

SimpleFutureValueMonthlyBFCL/simple_python_133 (finance.predict_future_value): present_value=5000, annual_interest_rate=0.05, compounding_periods_per_year=12, time_years=3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.annual_interest_rate == 0.05 Values: args.annual_interest_rate = 5

Failed

IN6 615 OUT2 537

✓

SimpleGeometryAreaTriangleBFCL/simple_python_104 (geometry.area_triangle): base=6, height=10, unit omitted or 'square meters'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 557 OUT861

✓

SimpleMathPowerBFCL/simple_python_105 (math.power): base=3, exponent=4, mod omitted or 1.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 550 OUT530

✓

SimpleNPVCashFlowsBFCL/simple_python_127 (calculate_NPV): cash_flows=[200,300,400,500], discount_rate=0.1, initial_investment=2000.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 665 OUT3 851

✓

SimpleReturnOnInvestmentBFCL/simple_python_135 (calculate_return_on_investment): purchase_price=20, sale_price=25, dividend=2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 593 OUT852

✓

SimpleStockPriceDataTypeBFCL/simple_python_146 (stock_price): company Amazon/AMZN, days=3, data_type omitted or Close.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 551 OUT1 372

✓

SimpleTwoSampleTTestFloatBFCL/simple_python_123 (hypothesis_testing.two_sample_t_test): group1=[12.4,15.6,11.2,18.9], group2=[10.5,9.8,15.2,13.8], alpha omitted or 0.05.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 759 OUT912

Tool Calling - Simple: Stats, Finance & ML (set 5) 8 passed

100.0%

IN53 410 OUT12 428

✓

SimpleAreaUnderCurvePolyBFCL/simple_python_103 (mathematics.calculate_area_under_curve): polynomial=[3.0,2.0,-4.0], limits=[-1.0,2.0].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 676 OUT2 090

✓

SimpleBinomialCalcBFCL/simple_python_115 (calculate_binomial_probability): number_of_trials=8, number_of_successes=5, probability_of_success omitted or 0.5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 644 OUT1 279

✓

SimpleCalculateBmiBFCL/simple_python_107 (calculate_bmi): weight=70, height=175, system omitted or metric.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 652 OUT990

✓

SimpleFutureValueInvestmentBFCL/simple_python_148 (finance.calculate_future_value): initial_investment=20000, rate_of_return=0.08, years=5, contribution omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 683 OUT1 303

✓

SimpleLinearRegressionBFCL/simple_python_108 (run_linear_regression): predictors=[Age,Income,Education], target=Purchase_Amount, standardize=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 671 OUT777

✓

SimpleNPVInvestmentBFCL/simple_python_130 (finance_calculator.npv): cash_flows=[-50000,10000,15000,20000,25000,30000], discount_rate=0.08, years omitted or empty.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 767 OUT2 570

✓

SimpleStockPricesDurationBFCL/simple_python_147 (get_stock_prices): companies=[Microsoft,Google], duration='2 weeks'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 621 OUT529

✓

SimpleTTestDatasetsBFCL/simple_python_124 (t_test): dataset_A=[12,24,36], dataset_B=[15,30,45], alpha omitted or 0.05.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 696 OUT2 890

Tool Calling - Simple: Stats, Finance & ML (set 6) 9 passed1 failed

90.0%

IN75 235 OUT16 452

✓

SimpleBinomialCoin100BFCL/simple_python_121 (calc_binomial_prob): num_trials=100, num_success=60, prob_success=0.5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 153 OUT2 165

✓

SimpleCagrBFCL/simple_python_140 (calculate_cagr): initial_value=2000, final_value=3000, period_in_years=4.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 172 OUT1 136

✓

SimpleChiSquaredTestBFCL/simple_python_122 (chi_squared_test): table=[[10,20],[30,40]], alpha omitted or 0.05.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 156 OUT2 240

✓

SimpleDiceRollBFCL/simple_python_113 (probability.dice_roll): desired_number=6, number_of_rolls=2, die_sides omitted or 6.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 130 OUT868

×

SimpleMarketPerformanceBFCL/simple_python_144 (market_performance.get_data): indexes=[S&P 500, Dow Jones], days=5, detailed omitted or true/false.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 891 OUT2 584

✓

SimpleMetalPriceBFCL/simple_python_141 (get_metal_price): metal=Gold, measure=ounce.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 096 OUT2 148

✓

SimpleNormalVariateBFCL/simple_python_111 (random.normalvariate): mu=0, sigma=1.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 117 OUT1 012

✓

SimplePortfolioFutureValueBFCL/simple_python_138 (portfolio_future_value): stock=X, invested_amount=5000, expected_annual_return=0.05, years=7.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 175 OUT1 014

✓

SimpleRandomForestClassifierBFCL/simple_python_106 (train_random_forest_classifier): dataset=your_dataset_name, max_depth=5, n_estimators=100.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 156 OUT466

✓

SimpleRSquaredBFCL/simple_python_126 (linear_regression.get_r_squared): dataset_path=C:/data/cars.csv, independent_variables=[engine_size,fuel_economy], dependent_variable=car_price.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 189 OUT2 819

Tool Calling - Tool Selection: Area (Triangle/Circle) 1 passed

100.0%

IN4 535 OUT988

✓

SelectTriangleAreaHeronToolBFCL/multiple_1 (math.triangle_area_heron): triangle area from three side lengths (Heron).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 535 OUT988

Tool Calling - Tool Selection: Art & Music 9 passed

100.0%

IN105 612 OUT12 022

✓

SelectAnalyzeStructureToolBFCL/multiple_162 (analyze_structure): dynamic structural analysis of specific floors.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN11 764 OUT953

✓

SelectConcertSearchToolBFCL/multiple_166 (concert.search): concert search by genre/location/date/price.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 730 OUT1 191

✓

SelectGuitarPriceToolBFCL/multiple_165 (guitar_price.find): guitar price by model/condition/location (not instrument_price.get).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN11 733 OUT1 526

✓

SelectInstrumentPriceToolBFCL/multiple_164 (instrument_price.get): instrument price by brand/model/finish (not guitar_price.find).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN11 738 OUT2 611

✓

SelectMelodyGeneratorToolBFCL/multiple_167 (music_generator.generate_melody): melody generation with key/note/length/tempo. Tightened per methodology: prompt explicitly gives "120 beats per minute", so tempo is required exactly 120 (ground truth also accepts omission, which equals the schema default).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 767 OUT789

✓

SelectMetropolitanArtworksToolBFCL/multiple_163 (metropolitan_museum.get_top_artworks): top artworks sorted by popularity.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 732 OUT1 200

✓

SelectMusicalScaleToolBFCL/multiple_169 (musical_scale): musical scale for a key.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 706 OUT885

✓

SelectSculptureInfoToolBFCL/multiple_160 (get_sculpture_info): sculpture info by artist with detail.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 717 OUT1 058

✓

SelectSongLyricsToolBFCL/multiple_168 (get_song_lyrics): lyrics by song title/artist.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 725 OUT1 809

Tool Calling - Tool Selection: Calculators & Directories 8 passed

100.0%

IN76 781 OUT11 061

✓

SelectCalculateGcdToolBFCL/multiple_35 (calculate_gcd): greatest common divisor, not lcm.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 591 OUT335

✓

SelectCurrencyGetRateToolBFCL/multiple_17 (currency_conversion.get_rate): rate at a specific date, not convert/latest-rate.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 610 OUT1 616

✓

SelectGeodistanceToolBFCL/multiple_65 (geodistance.find): distance between cities, not timezone difference/flights.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 580 OUT1 351

✓

SelectLibrarySearchBookToolBFCL/multiple_38 (library.search_book): search for a book, not reserve one.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 592 OUT1 217

✓

SelectLinearRegressionToolBFCL/multiple_53 (linear_regression): linear regression, not random forest.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 607 OUT1 136

✓

SelectMagneticFieldCalculateToolBFCL/multiple_41 (magnetic_field.calculate): magnetic field via Ampere's law, not electric field/force.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 623 OUT3 233

✓

SelectSolveQuadraticEquationToolBFCL/multiple_28 (solve.quadratic_equation): solve a quadratic, not rgb/string-reverse.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 599 OUT1 297

✓

SelectStockForecastToolBFCL/multiple_55 (stock_forecast): stock price prediction, not weather forecast.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 579 OUT876

Tool Calling - Tool Selection: Finance & Legal 7 passed3 failed

70.0%

IN215 466 OUT44 833

✓

SelectCagrCalculationToolBFCL/multiple_130 (calculate_cagr): compound annual growth rate from initial/final/years, not other finance tools.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 172 OUT1 008

×

SelectCaseInfoToolBFCL/multiple_135 (get_case_info): retrieve case info (victim) by docket/court.1 AI query 19 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 19

Failed

IN83 655 OUT29 791

✓

SelectCrimeRateToolBFCL/multiple_136 (get_crime_rate): crime rate by city/state/type/year.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 139 OUT1 531

✓

SelectCrimeRecordToolBFCL/multiple_134 (crime_record.get_record): detailed crime record by case number/county.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 140 OUT1 429

✓

SelectFutureValueToolBFCL/multiple_132 (finance.calculate_future_value): future value from rate/initial/years, not compound_interest.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 175 OUT1 799

×

SelectLawsuitDetailsToolBFCL/multiple_139 (lawsuit_details.find): patent lawsuit details by company/year/type.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN19 881 OUT2 436

×

SelectLawsuitSearchToolBFCL/multiple_137 (lawsuit_search): lawsuit search by company/date/location/status. Tightened per methodology: prompt explicitly asks for "ongoing" lawsuits, so status is required exactly "ongoing" (ground truth also accepts omission, but an explicitly-stated value must be passed).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN19 895 OUT1 604

✓

SelectLegalCaseFetchToolBFCL/multiple_138 (legal_case.fetch): fetch detailed legal case by id.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 112 OUT1 733

✓

SelectMarketPerformanceToolBFCL/multiple_131 (market_performance.get_data): index performance over N days.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 137 OUT2 335

✓

SelectMutualFundBalanceToolBFCL/multiple_133 (calculate_mutual_fund_balance): mutual fund balance from investment/yield/years.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 160 OUT1 167

Tool Calling - Tool Selection: Finance & Markets 8 passed

100.0%

IN82 450 OUT14 072

✓

SelectAvgClosingPriceToolBFCL/multiple_56 (avg_closing_price): average closing price, not volume/total revenue.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 275 OUT1 432

✓

SelectCompoundInterestCalcToolBFCL/multiple_49 (calc_Compound_Interest): compound interest, not future value/simple interest.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 324 OUT1 867

✓

SelectCurrencyConversionToolBFCL/multiple_52 (currency_conversion): currency conversion, not unit conversion.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 285 OUT3 187

✓

SelectDividendDataToolBFCL/multiple_54 (corporate_finance.dividend_data): dividend history, not generic stock market data.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 276 OUT1 858

✓

SelectHousePriceForecastToolBFCL/multiple_50 (house_price_forecast): house price forecast, not weather/stock forecast.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 269 OUT1 689

✓

SelectPropertyDepreciationToolBFCL/multiple_14 (finance.property_depreciation): depreciation, not loan/inflation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 324 OUT1 783

✓

SelectRevenueForecastToolBFCL/multiple_13 (corporate_finance.revenue_forecast): revenue forecast, not product price.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 316 OUT1 223

✓

SelectYoyGrowthRateToolBFCL/multiple_12 (corporate_finance.calculate_YOY_growth_rate): YOY growth, not ROE/ROA.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 381 OUT1 033

Tool Calling - Tool Selection: Food & Travel 6 passed1 failed

85.7%

IN76 168 OUT10 161

✓

SelectCurrencyConvertToolBFCL/multiple_95 (currency_exchange.convert): currency conversion, not unit_conversion. live_conversion is required true: prompt says "using current exchange rate" (ground truth = [true]).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 223 OUT1 321

✓

SelectDrivingDistanceToolBFCL/multiple_104 (get_shortest_driving_distance): driving distance, not instrument price / biology / scientist lookup.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 212 OUT928

✓

SelectHotelBookToolBFCL/multiple_93 (hotel.book): book a hotel room, not car.rental.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 241 OUT322

✓

SelectRecipeFindToolBFCL/multiple_91 (recipe.find): recipe by main ingredient + limit, not restaurant.find.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 210 OUT531

✓

SelectRecipeSearchToolBFCL/multiple_89 (recipe_search): find a recipe from ingredients, not restaurant_search/ingredient_replace.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 227 OUT1 384

✓

SelectRestaurantForGroupToolBFCL/multiple_90 (restaurant.find_group): restaurant for a group, not events.find_event.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 220 OUT898

×

SelectWalmartVeganToolBFCL/multiple_92 (walmart.vegan_products): the named store is Walmart, not Safeway/Whole Foods.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN20 835 OUT4 777

Tool Calling - Tool Selection: Games, Food & Utilities 8 passed2 failed

80.0%

IN137 974 OUT14 176

✓

SelectBestGroceryStoreToolBFCL/multiple_188 (grocery_store.find_best): best grocery store by rating/products.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 824 OUT884

✓

SelectClosestRestaurantToolBFCL/multiple_185 (restaurant_search.find_closest): closest restaurant by cuisine/amenities.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 789 OUT1 094

×

SelectFindRecipeToolBFCL/multiple_186 (find_recipe): recipe by restrictions/type/time.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.recipe_type.ToLower() == "dessert"

Failed

IN13 802 OUT1 521

✓

SelectGamePlayerStatsToolBFCL/multiple_180 (game_stats.fetch_player_statistics): player stats by game/username/platform.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 794 OUT546

×

SelectMultiplayerGameFinderToolBFCL/multiple_182 (multiplayer_game_finder): multiplayer game by platform/rating.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.rating == 4.5 Values: args.rating = 4.6

Failed

IN13 798 OUT1 656

✓

SelectRecipeCaloriesToolBFCL/multiple_183 (recipe_info.get_calories): calories of a recipe from a website.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 789 OUT3 828

✓

SelectRecipeSearchToolBFCL/multiple_184 (recipe_search): recipe by restriction/ingredients/servings.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 800 OUT1 432

✓

SelectSoccerLastMatchToolBFCL/multiple_181 (soccer.get_last_match): last match with stats for a club.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 793 OUT949

✓

SelectTimezoneConvertToolBFCL/multiple_189 (timezone.convert): convert a time between timezones.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 794 OUT1 285

✓

SelectWholeFoodsPriceToolBFCL/multiple_187 (whole_foods.check_price): price of items at a Whole Foods location.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 791 OUT981

Tool Calling - Tool Selection: Geometry, Finance & Services 7 passed

100.0%

IN58 975 OUT9 858

✓

SelectAirQualityForecastToolBFCL/multiple_63 (air_quality_forecast): air quality, not weather/news.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 409 OUT729

✓

SelectFinancialCompoundInterestToolBFCL/multiple_57 (financial.compound_interest): compound interest, not simple interest.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 472 OUT1 569

✓

SelectGeometryRectangleToolBFCL/multiple_31 (geometry_rectangle.calculate): rectangle area+perimeter, not square/circle.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 425 OUT1 082

✓

SelectLawyerFinderToolBFCL/multiple_59 (lawyer_finder): find a lawyer by specialization, not rent a car.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 406 OUT1 452

✓

SelectLibrarySearchBooksToolBFCL/multiple_68 (library.search_books): search a specific library, not Google Books/Open Library.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 409 OUT694

✓

SelectMapsShortestPathToolBFCL/multiple_27 (maps.shortest_path): shortest path with transport mode, not route-time estimation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 433 OUT3 087

✓

SelectTrafficEstimateToolBFCL/multiple_66 (traffic_estimate): traffic for a time period, not raw distance/weather.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 421 OUT1 245

Tool Calling - Tool Selection: History & Knowledge 9 passed1 failed

90.0%

IN136 015 OUT13 307

✓

SelectDiscovererToolBFCL/multiple_155 (get_discoverer): discoverer of a discovery, with detail.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 576 OUT830

×

SelectEarliestReferenceToolBFCL/multiple_157 (get_earliest_reference): earliest reference of a person from a source.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.source.ToLower() == "historical records"

Failed

IN13 578 OUT1 539

✓

SelectEventDateToolBFCL/multiple_153 (get_event_date): date of a historical event.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 571 OUT866

✓

SelectHistoricalContribToolBFCL/multiple_156 (historical_contrib.get_contrib): scientist's contribution on a date.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 608 OUT934

✓

SelectHistoryKeyEventsToolBFCL/multiple_152 (history.get_key_events): key war events by country/years.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 623 OUT2 611

✓

SelectMostFollowedToolBFCL/multiple_151 (social_media_analytics.most_followed): most-followed by topic/sub-topics.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 602 OUT1 934

✓

SelectNeuronalActivityToolBFCL/multiple_150 (calculate_neuronal_activity): neuronal activity from synaptic rate/weight/decay.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 632 OUT1 418

✓

SelectPaintNeededToolBFCL/multiple_159 (calculate_paint_needed): paint quantity for a wall.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 639 OUT687

✓

SelectPapalBiographyToolBFCL/multiple_158 (religious_history.get_papal_biography): papal biography with contributions.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 589 OUT1 624

✓

SelectUsPresidentToolBFCL/multiple_154 (US_president.in_year): president in a given year. Tightened per methodology: prompt explicitly asks for the "full name", so full_name is required true (ground truth also accepts omission, but an explicitly-requested option must be passed).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 597 OUT864

Tool Calling - Tool Selection: History, Demographics & Art 6 passed2 failed2 skipped

75.0%

IN97 063 OUT14 533

✓

SelectArtworkPriceToolBFCL/multiple_74 (art_auction.fetch_artwork_price): artwork price, not library book search. Tightened per methodology: prompt explicitly asks "on all auction platforms", so require platform == "all" (ground truth also accepts omission, but an explicitly-given value must pass).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 764 OUT1 368

✓

SelectArtworkSearchToolBFCL/multiple_77 (artwork_search.find): artwork search, not tourist_attraction/park search.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 747 OUT3 572

–

SelectCustomSculptureToolBFCL/multiple_76 (sculpture.create_custom): custom sculpture (material), not painting (color).0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Skipped: Ambiguous action-vs-question item (item flaw, not model behavior): the prompt mixes an order ('I want to order a custom bronze sculpture of a horse') with a question ('What material options are available?'), so a competent model may either call sculpture_create_custom or answer the question in text - not deterministic. Kept for traceability.

Skipped

IN0 OUT0

✓

SelectEuropeanMonarchsToolBFCL/multiple_70 (european_history.get_monarchs): monarch lookup, not events/culture.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 748 OUT801

×

SelectExhibitionInfoToolBFCL/multiple_79 (exhibition_info): museum exhibitions, not restaurant_info.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN16 286 OUT2 405

–

SelectMuseumInfoToolBFCL/multiple_78 (museum_info): museum info with a feature list, not generic tourist_spot_info.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Ambiguous single-call item (item flaw, not model behavior): a dedicated exhibition_info tool coexists with museum_info, so the 'exhibitions' part can reasonably route either to museum_info's features or to a separate exhibition_info call; a competent model may make one or two calls - not deterministic. Kept for traceability.

Skipped

IN0 OUT0

✓

SelectPaintColorTrendsToolBFCL/multiple_75 (paint_color.trends): paint color trend, not weather/house-price trends.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 732 OUT2 016

×

SelectPopulationByCategoryToolBFCL/multiple_71 (get_population): population by category, not bureau statistics/demographics.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN16 302 OUT2 350

✓

SelectReligionOriginToolBFCL/multiple_73 (religion.get_origin): founder/origin, not core beliefs.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 737 OUT1 034

✓

SelectStatePopulationToolBFCL/multiple_72 (us_history.population_by_state_year): state population, not GDP.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 747 OUT987

Tool Calling - Tool Selection: Isolated Collisions: Exhibition, Hotel & Currency 2 passed1 failed

66.7%

IN21 037 OUT3 779

✓

SelectBookLuxuryHotelToolBFCL/multiple_191 (book_hotel): luxury room with view, stay_duration variant.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 062 OUT531

✓

SelectCurrencyConversionToolBFCL/multiple_192 (currency_conversion.convert): convert an amount between currencies.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 977 OUT1 923

×

SelectExhibitionToolBFCL/multiple_161 (find_exhibition): top-rated sculpture exhibition by location/art_form.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.user_ratings == null) || (args.user_ratings?.ToLower() == "high") Values: args.user_ratings = "top"

Failed

IN6 998 OUT1 325

Tool Calling - Tool Selection: Knowledge & Media 9 passed1 failed

90.0%

IN123 755 OUT13 553

✓

SelectBattleDetailsToolBFCL/multiple_18 (european_history.battle_details): battle details, not war/leader info.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 747 OUT834

✓

SelectDatabaseModifyColumnsToolBFCL/multiple_10 (database.modify_columns): delete columns, not create a backup.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 804 OUT482

✓

SelectFiveFactorModelToolBFCL/multiple_69 (five_factor_model.analyse): Big Five personality, not MBTI.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 818 OUT1 206

✓

SelectGameRewardsToolBFCL/multiple_26 (game_rewards.get): rewards, not scores/missions list.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 744 OUT2 629

✓

SelectGenerateSoundWaveToolBFCL/multiple_21 (generate_sound_wave): generate a sound, not play a file.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 771 OUT554

✓

SelectMostPointsSingleGameToolBFCL/multiple_22 (sports_data.basketball.most_points_single_game): single-game record, not season/career.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 747 OUT574

✓

SelectPlayerStatsToolBFCL/multiple_23 (basketball.player_stats.get): player stats, not game/team stats.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 769 OUT887

✓

SelectReligionSchismsToolBFCL/multiple_19 (religion_history.get_schisms): schisms, not councils/reformations.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 739 OUT2 252

✓

SelectSculpturePriceToolBFCL/multiple_20 (sculpture_price.calculate): commission price, not sculptor info/availability.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 753 OUT1 433

×

SelectVideoGameStorePriceToolBFCL/multiple_25 (video_games.store_price): store price, not store currency/on-sale.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN17 863 OUT2 702

Tool Calling - Tool Selection: Legal, Environment & Services 9 passed1 failed

90.0%

IN127 872 OUT11 133

✓

SelectAirQualityToolBFCL/multiple_144 (air_quality): AQI by location/date.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 151 OUT1 113

✓

SelectEmissionsToolBFCL/multiple_145 (calculate_emissions): annual CO2 from distance/fuel/efficiency.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 187 OUT888

✓

SelectHumidityForecastToolBFCL/multiple_142 (weather.humidity_forecast): humidity forecast by location/days.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 138 OUT1 176

×

SelectLawsuitCheckCaseToolBFCL/multiple_141 (lawsuit.check_case): verify a case's closed status by id.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN18 403 OUT1 465

✓

SelectLawsuitsSearchToolBFCL/multiple_140 (lawsuits_search): lawsuits by company/location/year.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 152 OUT792

✓

SelectMapDirectionsToolBFCL/multiple_147 (map_service.get_directions): directions avoiding features.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN12 161 OUT879

✓

SelectRestaurantFindNearbyToolBFCL/multiple_146 (restaurant.find_nearby): nearby restaurants by cuisine/distance.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 148 OUT941

✓

SelectSentimentAnalysisToolBFCL/multiple_149 (sentiment_analysis): sentiment of a review text.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 158 OUT1 318

✓

SelectSlopeGradientToolBFCL/multiple_143 (calculate_slope_gradient): slope gradient between two coordinates. Tightened per methodology: prompt explicitly asks for the gradient "in degree", so unit is required exactly "degree" (ground truth also accepts omission, which equals the schema default).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN12 247 OUT1 404

✓

SelectStockInfoToolBFCL/multiple_148 (get_stock_info): detailed stock information by company.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 127 OUT1 157

Tool Calling - Tool Selection: Local Services & Weather 9 passed

100.0%

IN86 774 OUT10 727

✓

SelectHumidityTemperatureForecastToolBFCL/multiple_60 (humidity_temperature_forecast): humidity/temperature, not events/movies.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 632 OUT1 662

✓

SelectLandscapeArchitectToolBFCL/multiple_61 (landscape_architect.find_specialty): landscape architect, not home renovation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 649 OUT1 094

✓

SelectLawyerSearchToolBFCL/multiple_58 (lawyer.search): find a lawyer, not a doctor.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 618 OUT939

✓

SelectLibraryFindNearbyToolBFCL/multiple_48 (library.find_nearby): find a library, not a store.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 643 OUT1 094

✓

SelectNatureParkToolBFCL/multiple_62 (nature_park.find_nearby): find a nature park, not a restaurant.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 653 OUT1 666

✓

SelectRideHailingToolBFCL/multiple_39 (ride_hailing.get_rides): find a ride, not a grocery delivery.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 641 OUT1 925

✓

SelectRoutePlannerToolBFCL/multiple_24 (route_planner.calculate_route): plan a route, not chess-club lookup.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 657 OUT800

✓

SelectTranslateToolBFCL/multiple_67 (translate): translate text, not sentiment/word-count.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 634 OUT636

✓

SelectUvIndexToolBFCL/multiple_64 (uv_index.get_future): UV index, not rainfall/snowfall.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 647 OUT911

Tool Calling - Tool Selection: Mathematics 8 passed

100.0%

IN75 269 OUT9 028

✓

SelectCalculateIntegralToolBFCL/multiple_33 (calculate_integral): definite integral, not derivative.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 413 OUT1 363

✓

SelectCosineSimilarityToolBFCL/multiple_47 (cosine_similarity.calculate): cosine similarity, not correlation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 432 OUT2 053

✓

SelectDiceRollProbabilityToolBFCL/multiple_51 (dice_roll_probability): dice sum probability, not coin/card.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 407 OUT788

✓

SelectFunctionsIntersectToolBFCL/multiple_29 (functions.intersect): intersection of two functions, not zeros of one.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 409 OUT1 044

✓

SelectMathLcmToolBFCL/multiple_34 (math.lcm): least common multiple, not gcd/sqrt.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 399 OUT1 327

✓

SelectQuadraticRootsToolBFCL/multiple_11 (math_roots.quadratic): quadratic roots from coefficients, not cubic/polynomial.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 411 OUT1 229

✓

SelectRectangleAreaToolBFCL/multiple_30 (rectangle.area): rectangle area, not circle/triangle area.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 397 OUT583

✓

SelectSortListToolBFCL/multiple_46 (sort_list): sort a list, not filter/sum.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 401 OUT641

Tool Calling - Tool Selection: Mathematics & Geometry 7 passed

100.0%

IN70 323 OUT7 276

✓

SelectCircleAreaToolBFCL/multiple_97 (geometry.area_circle): circle area, not plot_sine_wave.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 026 OUT1 270

✓

SelectCircumferenceToolBFCL/multiple_98 (geometry.circumference): circle circumference, not the reference/time/melody tools.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 017 OUT806

✓

SelectDatabaseQueryToolBFCL/multiple_119 (database.query): structured DB query with conditions, not the score/ROI/probability distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN10 108 OUT1 179

✓

SelectDerivativeToolBFCL/multiple_99 (calculus.derivative): derivative at a point, not get_personality_traits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 050 OUT880

✓

SelectHcfToolBFCL/multiple_100 (math.hcf): highest common factor, not the scale-progression distractor.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 036 OUT747

✓

SelectSolveQuadraticToolBFCL/multiple_96 (solve_quadratic_equation): solve quadratic, not stock/portfolio tools.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 049 OUT1 606

✓

SelectTriangleAreaToolBFCL/multiple_121 (geometry.area_triangle): triangle area, not map directions / invention lookup.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 037 OUT788

Tool Calling - Tool Selection: Mixed Domains 7 passed1 skipped

100.0%

IN68 828 OUT8 656

✓

SelectAverageGradeToolBFCL/multiple_9 (calculate_average): selects average, not standard_deviation/highest_grade.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 853 OUT1 934

✓

SelectCapacitanceToolBFCL/multiple_6 (capacitance_calculator.calculate): selects capacitance, not resistance/magnetic_field. Tightened per methodology: prompt explicitly states "dielectric constant K is 1.0", so K is required exactly 1.0 (ground truth also accepts omission, but an explicitly-given value must be passed).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 855 OUT1 699

✓

SelectCapitalToolBFCL/multiple_2 (country_info.capital): selects capital lookup, not largest_city/population.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 754 OUT599

✓

SelectDisplacementToolBFCL/multiple_4 (kinematics.calculate_displacement): selects displacement, not final_speed.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 847 OUT1 109

✓

SelectEuclideanDistanceToolBFCL/multiple_3 (EuclideanDistance.calculate): selects distance, not angleToXAxis.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 799 OUT983

✓

SelectRealEstateSearchToolBFCL/multiple_8 (realestate.find_properties): property search with budget, not property_valuation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 856 OUT1 034

✓

SelectWeatherByCoordinatesDateToolBFCL/multiple_5 (weather.get_by_coordinates_date): historical-by-coordinates, not by-city/forecast.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 864 OUT1 298

–

SelectWildlifeGrowthToolBFCL/multiple_7 (wildlife_population.assess_growth): population growth, not ecological_impact.0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Skipped: Ambiguous single-call item (item flaw, not model behavior): the prompt asks for BOTH population growth AND impact on woodland, and two distinct tools genuinely fit (wildlife_population_assess_growth + ecological_impact_analyze); a competent human would reasonably make two calls, so a single-call ground truth is unfair. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Tool Selection: Music, Sports & Games 9 passed

100.0%

IN90 206 OUT11 265

✓

SelectBoardGameRecommendToolBFCL/multiple_86 (BoardGameGeek.recommend): BoardGameGeek store, not AmazonGameStore.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 019 OUT1 275

✓

SelectConcertTicketToolBFCL/multiple_81 (concert.book_ticket): concert ticket, not festival ticket.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 042 OUT404

✓

SelectGameUpdateToolBFCL/multiple_87 (games.update.find): game update/patch, not reviews/price.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 030 OUT1 069

✓

SelectMusicGenerateToolBFCL/multiple_82 (music.generate): musical piece from key/tempo, not raw audio.generate.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 008 OUT252

✓

SelectMusicShopToolBFCL/multiple_80 (music_shop.find_nearby): music shop with lessons, not gym.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 020 OUT1 225

✓

SelectPlayerCountToolBFCL/multiple_88 (video_games.get_player_count): active player count, not sales.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 027 OUT1 252

✓

SelectPlayerGoalsToolBFCL/multiple_83 (player_stats.get_all_time_goals): per-player goals, not team/league top scorer.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 008 OUT657

✓

SelectSoccerScoresToolBFCL/multiple_85 (soccer_scores.get_scores): soccer scores, not basketball.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 024 OUT2 780

✓

SelectTopGoalScorersToolBFCL/multiple_84 (getTopGoalScorers): top goal scorers, not top assists.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 028 OUT2 351

Tool Calling - Tool Selection: Physical Sciences 10 passed

100.0%

IN113 065 OUT22 454

✓

SelectBiologicalEnergyToolBFCL/multiple_43 (biological.calc_energy): energy from moles of substance, not biomass/work.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 291 OUT1 077

✓

SelectBiotSavartLawToolBFCL/multiple_40 (electromagnetism.biot_savart_law): magnetic field via Biot-Savart, not Ampere's law.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 294 OUT4 213

✓

SelectConeVolumeToolBFCL/multiple_32 (geometry.calculate_cone_volume): cone volume, not cone mass.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 274 OUT1 486

✓

SelectFinalTemperatureToolBFCL/multiple_42 (calculate_final_temperature): mixing two gas samples, not mass calculation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 335 OUT2 587

✓

SelectFinalVelocityToolBFCL/multiple_37 (kinematics.final_velocity): final velocity from initial velocity + time, not wave velocity/distance.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 284 OUT2 952

✓

SelectGeologyEraToolBFCL/multiple_45 (geology.get_era): geological era date, not historical event date.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 267 OUT1 750

✓

SelectPopulationGeneticsNeToolBFCL/multiple_16 (population_genetics.calculate_ne): effective population size, not range-shift/carrying-capacity.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 335 OUT2 038

✓

SelectSolarFarmPotentialToolBFCL/multiple_15 (solarFarm.potential): solar (not wind) energy potential from coordinates + panel area.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 390 OUT2 950

✓

SelectSpeedFromRestToolBFCL/multiple_36 (kinematics.calculate_speed_from_rest): speed from rest over a distance, not acceleration.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 310 OUT1 007

✓

SelectWeightInSpaceToolBFCL/multiple_44 (calculate.weight_in_space): weight on a planet, not unit/currency conversion.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 285 OUT2 394

Tool Calling - Tool Selection: Physics & Biology 13 passed

100.0%

IN195 030 OUT18 392

✓

SelectCellProteinsToolBFCL/multiple_109 (cellbio.get_proteins): proteins by cell compartment, not find_exhibition / mutation lookup.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN14 970 OUT1 952

✓

SelectDensityToolBFCL/multiple_107 (calculate_density): mass/volume density, not mix_paint_color.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN14 997 OUT621

✓

SelectDisplacementToolBFCL/multiple_102 (calculate_displacement): kinematic displacement, not poker/musical distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 015 OUT1 162

✓

SelectElectricFieldToolBFCL/multiple_106 (calculate_electric_field_strength): electric field, not paint/cooking/group distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN14 993 OUT1 881

✓

SelectEvolutionPredictionToolBFCL/multiple_114 (prediction.evolution): evolutionary-rate prediction, not modify_painting / calculate_probability.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 005 OUT1 125

✓

SelectFinalSpeedToolBFCL/multiple_103 (calculate_final_speed): free-fall final speed, not chess/court/event distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN14 985 OUT1 224

✓

SelectFitnessToolBFCL/multiple_113 (calculate_fitness): evolutionary fitness from trait values/contributions, not chess/walmart/lawyer distractors. Percentages map to decimal contributions [0.4, 0.6].1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN15 065 OUT1 110

✓

SelectForestGrowthToolBFCL/multiple_112 (forest_growth_forecast): forest growth forecast, not get_scientist_for_discovery. include_human_impact is required true: prompt says "including human impact" (ground truth = [true]).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN14 998 OUT1 402

✓

SelectGenotypeFrequencyToolBFCL/multiple_111 (calculate_genotype_frequency): genotype frequency, not factorial / find_flute.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 000 OUT914

✓

SelectHeatCapacityToolBFCL/multiple_108 (calc_heat_capacity): gas heat capacity, not discounted_cash_flow.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 016 OUT1 797

✓

SelectLightTravelTimeToolBFCL/multiple_120 (light_travel_time): light travel time, not grocery/hotel/exchange distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN14 983 OUT393

✓

SelectMagneticFieldToolBFCL/multiple_105 (calculate_magnetic_field): magnetic field of a loop, not concert/lawsuit distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN14 999 OUT2 825

✓

SelectMutationTypeToolBFCL/multiple_110 (mutation_type.find): SNP mutation type, not get_collectables_in_season.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 004 OUT1 986

Tool Calling - Tool Selection: Sports & Card Games 9 passed1 broken

90.0%

IN134 181 OUT75 900

✓

SelectChessTopPlayersToolBFCL/multiple_177 (board_game.chess.get_top_players): top chess players by location/rating.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 423 OUT1 155

✓

SelectDefenseRankingToolBFCL/multiple_173 (get_defense_ranking): NBA best-defence ranking by season.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 421 OUT1 671

✓

SelectFindAthleteToolBFCL/multiple_172 (sports_db.find_athlete): athlete profile by name/sport.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 406 OUT1 229

✓

SelectFindCardToolBFCL/multiple_178 (find_card_in_deck): find a card by rank/suit in default deck.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 417 OUT600

✓

SelectGameWinnerToolBFCL/multiple_171 (game_result.get_winner): winner of a game between two teams on a date.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 436 OUT2 335

✓

SelectPokerFullHouseToolBFCL/multiple_179 (poker_probability.full_house): probability of a full house (default deck/hand).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 407 OUT1 390

✓

SelectSoccerPlayerStatsToolBFCL/multiple_170 (soccer_stat.get_player_stats): player stats by name/season.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 434 OUT1 896

✓

SelectSportsRankingToolBFCL/multiple_174 (sports_ranking): team ranking in a league.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 404 OUT1 863

!

SelectTeamScheduleToolBFCL/multiple_176 (sports_team.get_schedule): team schedule for N games in a league.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Broken

IN13 428 OUT62 304

✓

SelectTopPlayerToolBFCL/multiple_175 (sports_ranking.get_top_player): top player by sport/gender.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 405 OUT1 457

Tool Calling - Tool Selection: Statistics, Finance & Health 5 passed1 failed

83.3%

IN56 328 OUT6 360

✓

SelectBmiMetersToolBFCL/multiple_117 (calculate_BMI): BMI in kg/m, not soccer / hotel_booking distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN9 367 OUT1 080

✓

SelectGcdToolBFCL/multiple_101 (math.gcd): greatest common divisor, not get_top_cases.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN9 344 OUT796

×

SelectHotelPricingToolBFCL/multiple_94 (hotel_room_pricing.get): hotel room pricing, not car/flight pricing.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.roomType.ToLower() == "suite with queen size bed"

Failed

IN9 375 OUT667

✓

SelectReturnOnEquityToolBFCL/multiple_128 (calculate_return_on_equity): ROE from income/equity/dividends, not library/lyrics/law distractors. dividends_paid is required 200000: prompt explicitly supplies "dividends paid of $200,000".1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 445 OUT1 070

✓

SelectSingleProbabilityToolBFCL/multiple_124 (probabilities.calculate_single): single-event probability, not math.power / fetch_DNA_sequence.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 353 OUT606

✓

SelectTwoSampleTtestToolBFCL/multiple_125 (run_two_sample_ttest): two-group t-test, not restaurant/personality distractors. equal_variance is required true: prompt says "assuming equal variance" (ground truth = [true, ""]).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 444 OUT2 141

Tool Calling - Tool Selection: Statistics, Finance & Services 8 passed

100.0%

IN91 264 OUT10 597

✓

SelectBmiMetricToolBFCL/multiple_116 (calculate_bmi): BMI in kg/cm with unit option, not celebrity_net_worth.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 395 OUT1 104

✓

SelectCalculateProbabilityToolBFCL/multiple_123 (calculate_probability): probability from favorable/total, not lawsuit/house-price distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 367 OUT876

✓

SelectCompoundInterestToolBFCL/multiple_129 (compound_interest): compound interest, not lawsuits_search.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 422 OUT1 927

✓

SelectFindRestaurantsToolBFCL/multiple_115 (find_restaurants): restaurant search with count + dietary, not instrument/stock/sports.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 400 OUT901

✓

SelectLinearRegressionToolBFCL/multiple_122 (run_linear_regression): regression with predictors, not recipe/itinerary/cooking distractors. standardize is required true: prompt says "Also apply standardization" (ground truth = [true]).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN11 417 OUT958

✓

SelectMoviesByActorToolBFCL/multiple_118 (imdb.find_movies_by_actor): movies by actor+year, not flight.book / lawsuits_search.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 398 OUT945

✓

SelectQuarterlyDividendToolBFCL/multiple_127 (finance.calculate_quarterly_dividend_per_share): dividend per share, not lyrics/paint/movie distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN11 421 OUT1 020

✓

SelectTTestToolBFCL/multiple_126 (t_test): two-dataset t-test, not event_finder / triangle-area distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 444 OUT2 866

Tool Calling - Tool Selection: Travel & Biology 7 passed1 failed

87.5%

IN97 927 OUT11 810

×

SelectBookHotelToolBFCL/multiple_190 (book_hotel): book a room by hotel/location/room_type/start/nights.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: args.start_date.AnyOf(["2022-12-10", "10/12/2022", "Dec.10,2022", "10th December 2022", "10 December 2022"]) Values: args.start_date = "December 10, 2022"

Failed

IN12 281 OUT1 186

✓

SelectCellProteinsToolBFCL/multiple_196 (cellbio.get_proteins): proteins in a cell compartment.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 212 OUT1 855

✓

SelectDistanceDurationToolBFCL/multiple_193 (maps.get_distance_duration): distance/duration between two places.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 231 OUT1 541

✓

SelectForestGrowthToolBFCL/multiple_199 (forest_growth_forecast): forest growth forecast with human impact.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 234 OUT1 543

✓

SelectGenotypeFrequencyToolBFCL/multiple_198 (calculate_genotype_frequency): genotype frequency from allele frequency.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 242 OUT1 316

✓

SelectHeatCapacityToolBFCL/multiple_195 (calc_heat_capacity): heat capacity for a gas by temp/volume. Tightened per methodology: prompt explicitly states the gas is "air", so gas is required exactly "air" (ground truth also accepts omission, which equals the schema default).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 258 OUT1 209

✓

SelectMuseumHoursToolBFCL/multiple_194 (get_museum_hours): museum opening hours on a day.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 223 OUT716

✓

SelectMutationTypeToolBFCL/multiple_197 (mutation_type.find): gene mutation type by SNP id.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 246 OUT2 444

Tool Calling - Tool Selection: Triangle Properties 1 passed

100.0%

IN4 638 OUT1 099

✓

SelectTrianglePropertiesToolBFCL/multiple_0 (triangle_properties.get): retrieves triangle dimensions/properties from 3 sides.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 638 OUT1 099

Qwen 3.6 35B NVFP4 Thinking | Benchmark #28

Tool Calling Benchmark based on BFCL

Results by difficulty

Results by category