Ai4Users - Qwen 3.6 27B FP8 Instruct

Benchmark items
Tool Calling - Irrelevance: Architecture & Artwork 9 passed	100.0%	IN31 151 OUT2 110
✓	NoToolForApartmentRentCostBFCL/irrelevance_158: apartment rental cost asked; only a new-construction cost estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 463 OUT98
✓	NoToolForBestJazzSingerBFCL/irrelevance_164: best Jazz singer asked; only a top-instrument-players search (singers are not instrument players) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 458 OUT474
✓	NoToolForCathedralCeilingDesignBFCL/irrelevance_157: how to design a cathedral ceiling asked; only a lookup of facts about an existing building is available (retrieves data, does not advise on design) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 459 OUT418
✓	NoToolForHolyBookDefinitionBFCL/irrelevance_144: definition of a holy book asked; only a search-within-a-named-holy-book tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 457 OUT82
✓	NoToolForLectureTeacherBFCL/irrelevance_167: teacher of upcoming lectures asked; only an upcoming-concert details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 460 OUT47
✓	NoToolForMostFamousSculptureBFCL/irrelevance_153: most famous sculpture asked; only a sculpting-tools lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 465 OUT253
✓	NoToolForPianoMaintenanceTipsBFCL/irrelevance_166: piano maintenance tips asked; only an instrument rental-prices lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 460 OUT437
✓	NoToolForScreamArtistBFCL/irrelevance_159: artist behind 'The Scream' asked. NOTE: the only tool (artwork_search) requires a museum_location that the user did not provide, so it cannot be correctly called; BFCL labels this irrelevant. Kept faithfully with ground truth = no tool call (flagged as borderline).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 464 OUT18
✓	NoToolForSculptureRockTypeBFCL/irrelevance_152: which rock suits a garden sculpture asked; only a sculpture 3D-model creator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 465 OUT283
Tool Calling - Irrelevance: Buildings, Sculpture & Music 7 passed1 skipped	100.0%	IN23 003 OUT1 220
–	NoToolForCelloInstrumentTypeBFCL/irrelevance_165: instrument classification of a cello asked. NOTE: BFCL labels this irrelevant, but the only tool (get_instrument_info with detail=type) arguably answers it; flagged for review as a borderline BFCL item. Kept faithfully with ground truth = no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_instrument_info' with detail='type' answers 'what type of instrument is a cello' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForDeclarationSignersBFCL/irrelevance_137: who signed the declaration asked; only a historical-event DATE lookup is available (returns a date, not the signatories) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 284 OUT74
✓	NoToolForEiffelTowerHeightBFCL/irrelevance_156: Eiffel Tower height asked; only an architecture-plan generator (creates a new design, does not retrieve facts) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 288 OUT32
✓	NoToolForFamousDancersBFCL/irrelevance_162: famous dancers asked; only a museum-visual-artists lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 290 OUT457
✓	NoToolForReformationInitiatorBFCL/irrelevance_145: who initiated the Reformation asked; only a religious-event YEAR lookup is available (returns a year, not a person) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 283 OUT80
✓	NoToolForScientistDefinitionBFCL/irrelevance_143: definition of "scientist" asked; only a named-historical-figure info lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 280 OUT167
✓	NoToolForSellingGuitarBFCL/irrelevance_163: how to sell a guitar asked; only an instrument-tuning tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 285 OUT247
✓	NoToolForSimilarBlueColorBFCL/irrelevance_148: a color similar to blue asked; only a complementary-color tool (opposite, not similar) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 293 OUT163
Tool Calling - Irrelevance: Cell Biology & Genetics 8 passed2 skipped	100.0%	IN30 331 OUT2 070
✓	NoToolForClottingBloodCellsBFCL/irrelevance_32: blood cell type responsible for clotting asked; only a nucleus-count-based cell-type classifier is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 792 OUT50
✓	NoToolForCommonAncestorBFCL/irrelevance_41: common ancestor of two animals asked (phylogenetics knowledge); only a fibonacci-sequence calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 791 OUT209
✓	NoToolForEvolutionaryAlgorithmInGameAIBFCL/irrelevance_43: how to apply an Evolutionary Algorithm in game AI asked (advice); only a creature-evolution simulator is available (wrong context).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 792 OUT525
✓	NoToolForGeneVariantFrequencyBFCL/irrelevance_36: gene-variant frequency in a population asked; only a dominant-trait-from-alleles tool is available (the true-match variant-frequency tool is deliberately isolated in A4).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 797 OUT194
–	NoToolForIdentifyingGeneticSequenceBFCL/irrelevance_33: interpret a genetic code sequence asked; only a species-identification-from-sequence tool is available (returns a species, not what the sequence means).0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'identify_species' (identifies a species from a genetic sequence) plausibly answers the query about a DNA sequence, so a competent model may call it - BFCL-mislabeled/borderline-relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForIncreasingDeerPopulationBFCL/irrelevance_39: how-to advice on growing an animal population asked; only a population-growth CALCULATOR (needs numeric rates) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 791 OUT246
✓	NoToolForLionMatingProcessBFCL/irrelevance_35: mating process of a species asked; only a dominant/recessive genetic-traits lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 788 OUT255
✓	NoToolForMitochondriaCountBFCL/irrelevance_29: number/count of mitochondria asked; only a cell-part FUNCTION lookup is available (wrong query direction).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 792 OUT220
✓	NoToolForPandaEvolutionaryHistoryBFCL/irrelevance_42: evolutionary history asked (a narrative fact); only a biodiversity-index calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 788 OUT371
–	NoToolForWhaleGeneSequenceBFCL/irrelevance_44: the actual gene sequence of a species asked; only a gene-sequence GENERATOR (needs numeric mutation rate + duration) is available.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'gene_sequencer' (generate gene sequences for evolutionary changes) directly answers the query - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
Tool Calling - Irrelevance: Color, Geometry & Population 11 passed	100.0%	IN40 858 OUT1 271
✓	NoToolForColorInComputerVisionBFCL/irrelevance_131: color meaning in computer vision asked; only a color-in-personality-psychology analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 714 OUT253
✓	NoToolForDeclarationSigningDateBFCL/irrelevance_135: signing date asked; only date-arithmetic (add days) and age tools are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 712 OUT86
✓	NoToolForElectricityDiscovererBFCL/irrelevance_140: who discovered electricity asked; only an electric-current calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 708 OUT260
✓	NoToolForHexColorMixBFCL/irrelevance_146: mixing two hex colors asked; only a prophet-details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 724 OUT191
✓	NoToolForIndependenceSigningDateBFCL/irrelevance_138: signing date asked; only an age calculator (needs birthdate + current date) and a date-adder are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 712 OUT68
✓	NoToolForMostVisitedMarketBFCL/irrelevance_161: most visited market asked; only a museum visitation-statistics lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 714 OUT79
✓	NoToolForNbaFinalWinnerBFCL/irrelevance_133: NBA final winner asked; only a historical-battle details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 715 OUT36
✓	NoToolForPicassoPaintingYearBFCL/irrelevance_151: year of a Picasso painting asked; only a sculpture-dimensions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 713 OUT78
✓	NoToolForRelativityAuthorBFCL/irrelevance_142: who proposed special relativity asked; only a chemical-element properties lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 717 OUT42
✓	NoToolForSculptureCreatorBFCL/irrelevance_155: sculpture creator asked; only a max-seating-capacity calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 713 OUT13
✓	NoToolForWhichColorsToMixBFCL/irrelevance_150: which colors to mix for a shade asked; only an RGB-to-Pantone converter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 716 OUT165
Tool Calling - Irrelevance: Cooking, Currency & Markets 7 passed1 failed	87.5%	IN29 247 OUT2 131
×	NoToolForBakerySourdoughBFCL/irrelevance_223: a BAKERY selling sourdough asked; only a grocery-shop product finder available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 535 OUT299
✓	NoToolForChickenRecipeBFCL/irrelevance_211: the RECIPE to cook chicken asked; only a cook-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 243 OUT273
✓	NoToolForChocolateCakeIngredientsBFCL/irrelevance_220: cake INGREDIENTS asked; only a cooking-TEMPERATURE tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 246 OUT124
✓	NoToolForCurrencyToGbpBFCL/irrelevance_233: CURRENCY conversion asked; only a WEATHER tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 248 OUT88
✓	NoToolForHowToBoilAnEggBFCL/irrelevance_212: best WAY to boil an egg asked; only a boiling-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 245 OUT201
✓	NoToolForLasagnaRecipeBFCL/irrelevance_219: the RECIPE for lasagna asked; only a recipe-CALORIE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 243 OUT539
✓	NoToolForLegExercisesBFCL/irrelevance_221: leg EXERCISES asked; only a grocery-list generator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 243 OUT449
✓	NoToolForPetStoreBFCL/irrelevance_224: a PET store asked; only a GROCERY-store locator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 244 OUT158
Tool Calling - Irrelevance: Dates & Elements 9 passed1 failed	90.0%	IN40 022 OUT1 554
✓	NoToolForHydrogenPropertiesBFCL/irrelevance_141: chemical properties of Hydrogen asked; only a scientist-contributions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 628 OUT319
✓	NoToolForLargestPlanetBFCL/irrelevance_139: largest planet asked; only a star-info lookup (needs a star name) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 628 OUT143
✓	NoToolForMostImportantProphetBFCL/irrelevance_147: most important prophet asked; only a two-color mixer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 628 OUT101
×	NoToolForMuseumVisitFrequencyBFCL/irrelevance_160: museum member visit frequency asked; only a single most-frequent-visitor lookup is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN7 365 OUT292
✓	NoToolForPantoneOfSkyBlueBFCL/irrelevance_149: Pantone code of sky blue asked; only a paint-proportions calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 630 OUT143
✓	NoToolForPersonalityTypeBFCL/irrelevance_130: user asks what personality type they are; only a big-five calculator (which needs the user's rated answers) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 626 OUT87
✓	NoToolForStadiumSeatingCapacityBFCL/irrelevance_154: stadium seating capacity asked; only a sculpture-creator lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 629 OUT97
✓	NoToolForVicePresidentBFCL/irrelevance_136: Vice President asked; only a US-president-by-year lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 628 OUT46
✓	NoToolForWaterlooCasualtiesBFCL/irrelevance_132: casualty count of a battle asked; only a historical-event date lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 630 OUT240
✓	NoToolForWorldCupWinnerBFCL/irrelevance_134: sports World Cup winner asked; only a historical-battle outcome predictor is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 630 OUT86
Tool Calling - Irrelevance: Electromagnetism, Thermo & Data 10 passed	100.0%	IN34 065 OUT1 355
✓	NoToolForElectronRestMassBFCL/irrelevance_23: mass of an electron asked (a knowledge fact); only magnetic-field / current tools are available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 399 OUT46
✓	NoToolForFlightDurationBFCL/irrelevance_20: flight duration asked; only a currency converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 402 OUT146
✓	NoToolForFreezingPointBFCL/irrelevance_25: freezing point asked; only boiling-point calculators are available (wrong phase transition).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 409 OUT152
✓	NoToolForGasAmountGeneratedBFCL/irrelevance_26: amount of gas generated asked; only a gas-PRESSURE calculator is available (wrong quantity).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 422 OUT96
✓	NoToolForGiniCoefficientBFCL/irrelevance_48: Gini coefficient asked; only a GDP-fetch tool is available (different economic indicator).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 402 OUT231
✓	NoToolForHeatingWaterBFCL/irrelevance_27: heat energy to warm water asked. NOTE: the original tool (calculate_heat) is a fair match for this query (only the well-known specific heat is missing), which would make "no call" ambiguous; per human-baseline fairness it is deliberately NOT shown here (kept as a distractor in A3), so no visible tool fits and "no call" is unambiguously correct.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 413 OUT315
✓	NoToolForHexagonSidesBFCL/irrelevance_28: number of sides of a hexagon asked (basic geometry knowledge); only a boiling-point tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 400 OUT9
✓	NoToolForSymbolicMagneticFieldBFCL/irrelevance_22: magnetic field asked with SYMBOLIC values (distance 'r', current 'I'); the two magnetic-field tools require concrete numeric distance + current, so there is nothing to pass -> no fit.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 413 OUT150
✓	NoToolForTimeDifferenceBFCL/irrelevance_49: time-zone difference asked; only a geo-coordinate lookup tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 402 OUT184
✓	NoToolForWorldSeriesWinnerBFCL/irrelevance_53: sports trivia asked; only a SQL database-query runner is available (no database/query provided).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 403 OUT26
Tool Calling - Irrelevance: Entertainment, Sports & Knowledge 9 passed	100.0%	IN31 428 OUT1 423
✓	NoToolForBasketballResultBFCL/irrelevance_84: basketball game result asked; only a stock-price retrieval tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 492 OUT58
✓	NoToolForBestSoccerPlayerBFCL/irrelevance_79: best current soccer player asked; only a stock-market prediction tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 490 OUT356
✓	NoToolForCaliforniaGovernorBFCL/irrelevance_87: current Governor of California asked; only a court-case filing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 488 OUT41
✓	NoToolForCrimeThrillerMovieListBFCL/irrelevance_88: best crime-thriller movies of 2020 asked; only a document forgery detector is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 497 OUT203
✓	NoToolForFifaWorldCupWinnerBFCL/irrelevance_80: FIFA World Cup 2010 winner asked; only a stock-ticker lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 493 OUT48
✓	NoToolForFrenchTranslationBFCL/irrelevance_61: French translation of a word asked; only an SVM-model tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 492 OUT98
✓	NoToolForHorrorMovieListBFCL/irrelevance_77: horror movie recommendations asked; only an investment future-value tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 491 OUT413
✓	NoToolForPresidentialElectionWinnerBFCL/irrelevance_85: 2020 presidential election winner asked; only a criminal-case detail lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 493 OUT40
✓	NoToolForSportsNewsBFCL/irrelevance_76: latest sports news asked; only a portfolio investment-trend analyzer is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 492 OUT166
Tool Calling - Irrelevance: Finance & Economics 8 passed	100.0%	IN26 842 OUT1 313
✓	NoToolForAverageHousePriceBFCL/irrelevance_69: market-wide average house price asked; only a regression predictor (needs a trained model + feature vector, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 348 OUT124
✓	NoToolForCompoundInterestBFCL/irrelevance_70: compound-interest computation asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 374 OUT492
✓	NoToolForGoldPriceBFCL/irrelevance_78: today's gold spot price asked; only a bond-price calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 350 OUT84
✓	NoToolForHorrorMovieCatalogBFCL/irrelevance_81: horror movie recommendations asked; only a historical stock-price fetcher is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 351 OUT336
✓	NoToolForLossProjectionBFCL/irrelevance_73: a loss projection asked; only a revenue-prediction tool is available (revenue != loss).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 353 OUT102
✓	NoToolForPoundToKilogramBFCL/irrelevance_75: a mass unit conversion asked; only a portfolio-returns calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 349 OUT14
✓	NoToolForProfitMarginBFCL/irrelevance_71: profit margin asked; only an ROI calculator is available (ROI != margin) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 371 OUT109
✓	NoToolForStockNewsBFCL/irrelevance_82: stock news asked; only a capital-gains calculator is available (gains != news) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 346 OUT52
Tool Calling - Irrelevance: Finance Near-Matches 8 passed1 failed	88.9%	IN34 473 OUT1 843
✓	NoToolForBitcoinPriceBFCL/irrelevance_56: current bitcoin price asked; only a generic database record-select tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 428 OUT66
✓	NoToolForBookstoreLookupBFCL/irrelevance_55: which online bookstore sells a title asked; only an add-to-cart tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 433 OUT89
×	NoToolForBusinessRateOfReturnBFCL/irrelevance_74: rate of return asked; only a net-profit calculator is available (profit != rate of return).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 983 OUT321
✓	NoToolForCoinFlipProbabilityBFCL/irrelevance_66: probability of a coin landing heads asked; only a standard-deviation tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 433 OUT63
✓	NoToolForCurrentInterestRateBFCL/irrelevance_83: the current market interest rate asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 427 OUT122
✓	NoToolForExternalRateOfReturnBFCL/irrelevance_72: EXTERNAL rate of return asked; only an internal-rate-of-return (IRR) tool is available (ERR != IRR, even though cash flows are given) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 457 OUT234
✓	NoToolForFinalPriceWithoutBasePriceBFCL/irrelevance_60: final price after discount+tax asked, but the original price is never given, so the (domain-matching) calculateFinalPrice cannot be correctly called -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 447 OUT52
✓	NoToolForFrontendDevelopmentBFCL/irrelevance_62: how to build an e-commerce frontend asked; only a recommender-model builder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 433 OUT861
✓	NoToolForHighestGrossingMovieBFCL/irrelevance_54: highest-grossing movie of all time asked; only a movie search (requires a specific title + year, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 432 OUT35
Tool Calling - Irrelevance: Food, Time & Distance 4 passed1 skipped	100.0%	IN10 840 OUT364
✓	NoToolForCakeBakingMachineBFCL/irrelevance_218: a baking MACHINE/appliance asked; only a recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 708 OUT37
✓	NoToolForDrivingTravelTimeBFCL/irrelevance_215: driving TRAVEL TIME asked; only a restaurant-bill tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 712 OUT148
✓	NoToolForLondonTimezoneNameBFCL/irrelevance_226: the NAME of London's timezone asked; only a local-TIME-by-timezone tool available (it needs the timezone as input and returns a time, not the timezone name) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 704 OUT54
–	NoToolForPizzaInBostonBFCL/irrelevance_213: a place for pizza asked; a cuisine+location restaurant_finder is available. REVIEW FLAG (fairness): restaurant_finder(cuisine=pizza/Italian, location=Boston) plausibly answers this, so a competent human might legitimately call it. BFCL labels it irrelevant; kept as such and marked Hard because the intended "no call" is debatable. No OTHER query here is a restaurant search, so the tool cannot wrongly satisfy the file's other cases.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'restaurant_finder' (cuisine+location) answers 'good place for pizza in Boston' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForTriangleHypotenuseBFCL/irrelevance_238: the HYPOTENUSE of a triangle asked; only a geo lat/long distance calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 716 OUT125
Tool Calling - Irrelevance: Geo, Travel & Sentiment 8 passed1 skipped	100.0%	IN27 187 OUT930
✓	NoToolForAverageTemperatureBmiBFCL/irrelevance_110: an average December temperature asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 398 OUT51
✓	NoToolForCaseLawyerBFCL/irrelevance_95: the lawyer of a case asked; only a case-outcome retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 401 OUT106
✓	NoToolForCurrentTimeSunriseBFCL/irrelevance_102: the current time asked; only a sunrise/sunset time calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 397 OUT102
✓	NoToolForFamousQuoteBFCL/irrelevance_119: a famous quote asked; only a sentiment-analysis tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 395 OUT30
✓	NoToolForFastestRouteBFCL/irrelevance_114: the fastest route asked; only a closest-airport finder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 399 OUT126
–	NoToolForRestaurantsInLondonBFCL/irrelevance_113: restaurants in London asked; only a generic points-of-interest locator is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'find_pois' (locate points of interest by category) answers 'restaurants in London' (restaurants are POIs) - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForSportsBrainImpactBFCL/irrelevance_120: the neurological impact of sports asked; only a caffeine neurological-impact tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 398 OUT246
✓	NoToolForTrafficConditionBFCL/irrelevance_112: the current traffic condition asked; only an address geocoder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 397 OUT114
✓	NoToolForTravelTimeByCarBFCL/irrelevance_115: travel time by car asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 402 OUT155
Tool Calling - Irrelevance: Legal & Case Lookup 10 passed	100.0%	IN35 527 OUT1 197
✓	NoToolForAppleCaloriesBFCL/irrelevance_100: calories in an apple asked; only a litigation-cost calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 550 OUT94
✓	NoToolForBrainWeightBFCL/irrelevance_122: average human brain weight asked; only a per-patient neural-activity tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 552 OUT101
✓	NoToolForCaseJudgementBFCL/irrelevance_93: a case judgement asked; only a law-firm impactful-cases lister is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 550 OUT59
✓	NoToolForChineseRestaurantBFCL/irrelevance_97: a Chinese-restaurant recommendation asked; only a lawsuit-filing tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 553 OUT182
✓	NoToolForMovieRecommendationBFCL/irrelevance_116: a movie recommendation asked; only a word-count tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 551 OUT102
✓	NoToolForNbaChampionBFCL/irrelevance_118: the 1996 NBA champion asked; only an NBA playoff-brackets display tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 553 OUT48
✓	NoToolForPaintingEiffelSettlementBFCL/irrelevance_98: how long to paint the Eiffel Tower asked; only a lawsuit-settlement estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 554 OUT147
✓	NoToolForSecurityGuardSearchBFCL/irrelevance_92: finding a security guard asked; only a lawyer-search tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 560 OUT53
✓	NoToolForSphereVolumeBFCL/irrelevance_104: sphere volume asked; only a circular-park area tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 554 OUT176
✓	NoToolForTrafficLawsBFCL/irrelevance_99: traffic laws asked; only a lawsuit-search tool is available (lawsuits, not laws) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 550 OUT235
Tool Calling - Irrelevance: Math & Science Distractors 8 passed	100.0%	IN25 252 OUT1 089
✓	NoToolForDefiniteIntegralBFCL/irrelevance_9: definite integral asked; only an income-tax calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 159 OUT198
✓	NoToolForIntegralOfCubeBFCL/irrelevance_8: definite integral asked; only a string-to-int converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 152 OUT198
✓	NoToolForLatestMoviesBFCL/irrelevance_15: latest movie releases asked; only a physics final-velocity tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 146 OUT106
✓	NoToolForPerpendicularSlopeBFCL/irrelevance_3: slope of a perpendicular line asked; only a critical-points finder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 164 OUT147
✓	NoToolForPrimeFactorsBFCL/irrelevance_13: prime factorization asked; only a compound-interest calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 149 OUT97
✓	NoToolForRectangleAreaBFCL/irrelevance_6: rectangle area asked; only a circle-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 160 OUT74
✓	NoToolForTriangleAreaBFCL/irrelevance_0: triangle area asked; only a BMI tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 160 OUT82
✓	NoToolForUpwardAccelerationBFCL/irrelevance_14: acceleration asked (it is just gravity); the tempting max-height and final-velocity tools compute neither the acceleration nor what is asked -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 162 OUT187
Tool Calling - Irrelevance: Math & Statistics 6 passed	100.0%	IN18 082 OUT1 298
✓	NoToolForBinomialMeanBFCL/irrelevance_67: the MEAN of a binomial experiment asked; only p-value tools are available (they return a p-value, not the mean) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 028 OUT68
✓	NoToolForCivilLawComponentsBFCL/irrelevance_90: the components/categories of Civil Law asked; only a noise-complaint filing tool is available (filing a complaint does not explain law) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 000 OUT214
✓	NoToolForCoinTossMaxHeadsBFCL/irrelevance_63: how many heads CAN be obtained from tossing 3 coins (a count/maximum, not a probability) asked; only a probability-of-an-event calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 004 OUT150
✓	NoToolForNullHypothesisStdDevBFCL/irrelevance_68: the STANDARD DEVIATION of a null-hypothesis test asked; only p-value tools are available (they return a p-value, not a standard deviation) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 038 OUT549
✓	NoToolForPowerOfTwoBFCL/irrelevance_59: raising a number to a power asked; only a linear-equation solver is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 003 OUT20
✓	NoToolForQuadraticEquationBFCL/irrelevance_57: solving a quadratic equation asked; only a genetic-algorithm optimizer and a LINEAR equation solver are available (a quadratic is not linear) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 009 OUT297
Tool Calling - Irrelevance: Medical, Social & Personality 6 passed3 skipped	100.0%	IN20 464 OUT1 445
✓	NoToolForBigMacCaloriesBFCL/irrelevance_123: calories of a Big Mac asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 409 OUT111
✓	NoToolForEconomicHappinessEffectBFCL/irrelevance_126: the effect of economic status on happiness asked; only a happiness-index lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 411 OUT296
✓	NoToolForFriendMbtiBFCL/irrelevance_129: a friend's MBTI asked; only a personality-type evaluator (needs trait percentages) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 409 OUT140
–	NoToolForLatestTechTrendBFCL/irrelevance_124: the latest technology trend asked; only a trending-topics-by-category tool is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_social_trends' (trending topics by category) answers 'latest trend in technology' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForMajorAirportsBFCL/irrelevance_117: major US airports asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 412 OUT328
–	NoToolForMotorNeuronDiseaseBFCL/irrelevance_121: motor neuron disease info asked; only a generic disease-info retriever is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'medical_records_get_disease_info' directly answers 'information on motor neuron diseases' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForPopularBooksBFCL/irrelevance_125: popular books by an author asked; only a recent-tweets retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 411 OUT438
–	NoToolForTwitterMoodBFCL/irrelevance_127: overall twitter mood asked; only a twitter sentiment-analysis tool is available -> BFCL ground truth is no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'sentiment_analysis_twitter' directly answers 'general mood of twitter regarding the new iPhone' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForVegetableServingsBFCL/irrelevance_128: daily vegetable servings asked; only a personality-test scorer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 412 OUT132
Tool Calling - Irrelevance: Mixed Distractors 8 passed	100.0%	IN26 008 OUT1 249
✓	NoToolForAirQualityBFCL/irrelevance_40: current air quality asked; only a plant-biomass calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 252 OUT118
✓	NoToolForFavoriteRestaurantBFCL/irrelevance_31: a favorite restaurant recommendation asked; only a cell-division simulator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 248 OUT107
✓	NoToolForGlobalWarmingDocumentaryBFCL/irrelevance_38: a documentary (video) asked; only a scientific-research-paper retriever is available (wrong medium).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 249 OUT196
✓	NoToolForLasagnaIngredientsBFCL/irrelevance_47: recipe ingredients asked; only a flight-timetable tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 249 OUT239
✓	NoToolForLionDominantTraitBFCL/irrelevance_34: dominant genetic trait of a species asked; only a gene-variant-FREQUENCY lookup is available (different genetic output).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 251 OUT227
✓	NoToolForMultinucleatedCellNameBFCL/irrelevance_30: name of a cell type with multiple nuclei asked; only a blood-cell classifier (by shape/size) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 255 OUT88
✓	NoToolForPicnicSpotBFCL/irrelevance_37: a picnic spot asked; only a local-fauna information tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 248 OUT245
✓	NoToolForTranslationBFCL/irrelevance_52: natural-language translation asked; only a file-compression tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 256 OUT29
Tool Calling - Irrelevance: Motion, Electricity & Misc 10 passed	100.0%	IN35 428 OUT1 915
✓	NoToolForCarProjectileRangeBFCL/irrelevance_16: projectile-style "how far will a car travel" asked with symbolic v/t/theta; the range tool needs concrete numeric velocity + angle -> nothing to pass -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 553 OUT421
✓	NoToolForConeVolumeBFCL/irrelevance_19: cone volume asked; only an investment-return calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 547 OUT198
✓	NoToolForCurrentClockTimeBFCL/irrelevance_17: current clock time asked; only a travel-time (distance/speed) calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 536 OUT35
✓	NoToolForElectronMassBFCL/irrelevance_24: mass of an electron asked (a knowledge fact); only an electric-current tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 537 OUT46
✓	NoToolForFibonacciOf20BFCL/irrelevance_51: fibonacci number asked; only a cryptocurrency-price tool is available (the fibonacci calculator lives in a different file so it cannot satisfy this query).1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 538 OUT325
✓	NoToolForForceAngleFromMomentumBFCL/irrelevance_18: angle of a force from momentum asked; only a vector-angle-from-XY-components tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 543 OUT206
✓	NoToolForHorrorMovieListBFCL/irrelevance_50: movie recommendations asked; only a Celsius-to-Fahrenheit converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 543 OUT342
✓	NoToolForMagneticFieldNearWireBFCL/irrelevance_21: magnetic field near a current-carrying wire asked; only an EM-wave-amplitude tool is available (magnetic-field calculators are deliberately kept in a different file).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 552 OUT250
✓	NoToolForSineOf45DegreesBFCL/irrelevance_45: sine of an angle asked; only a polygon-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 538 OUT30
✓	NoToolForTeslaPriceBFCL/irrelevance_46: product price of a car asked; only a currency-exchange-rate tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 541 OUT62
Tool Calling - Irrelevance: Music Theory & Lookup 8 passed	100.0%	IN25 588 OUT1 136
✓	NoToolForAncientGreekLawmakersBFCL/irrelevance_171: who established laws in Ancient Greece asked (history); only a musical-piece composer search is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 196 OUT248
✓	NoToolForBasketballGameAtVenueBFCL/irrelevance_169: who is playing a basketball game at a venue asked; only concert-locating tools exist.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 199 OUT95
✓	NoToolForBluesScalePatternBFCL/irrelevance_176: the interval pattern of a blues scale asked; only composer lookups are available (get_blues_scale lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 201 OUT136
✓	NoToolForDonQuixoteAuthorBFCL/irrelevance_172: author of the novel Don Quixote asked; only music-composition lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 194 OUT54
✓	NoToolForFamousUsComposersBFCL/irrelevance_170: most famous US composers asked (general knowledge, no specific piece); only a chord-progression creator + piece-specific composer lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 197 OUT387
✓	NoToolForGrammyBestAlbumBFCL/irrelevance_177: Grammy Award winner asked; only a song chord-progression lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 202 OUT24
✓	NoToolForPrimaryTriadsOfKeyBFCL/irrelevance_173: primary triads of a key asked; only a genre-common-chords finder is available (primary_triads lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 200 OUT88
✓	NoToolForUniversityClassAvailabilityBFCL/irrelevance_168: availability of a university class asked; only concert-availability / concert-search tools are available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 199 OUT104
Tool Calling - Irrelevance: Probability & Legal 5 passed	100.0%	IN14 007 OUT890
✓	NoToolForFaceCardProbabilityBFCL/irrelevance_64: probability of drawing a face card from a deck asked; only a COIN-toss heads probability tool is available (coins, not cards) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 798 OUT88
✓	NoToolForHomeElectricityNeedBFCL/irrelevance_58: home electricity demand (kWh) asked; only a solar-panel COUNT calculator is available (it sizes panels, not electricity need, and needs sunlight hours not given) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN2 801 OUT194
✓	NoToolForNoiseComplaintProcessBFCL/irrelevance_91: how/whether to report a noise complaint asked; only a law-category listing tool is available (it lists legal categories, it does not file or advise on complaints) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 798 OUT188
✓	NoToolForRedMarbleCountBFCL/irrelevance_65: number of red marbles asked; only a determine-population tool is available (it returns a total population from probability + sample size, not a sub-category count) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN2 813 OUT76
✓	NoToolForTenantRightsBFCL/irrelevance_89: explaining tenant rights asked; only a contract-generation tool is available (generating a contract does not explain rights) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN2 797 OUT344
Tool Calling - Irrelevance: Recipes & Cooking 8 passed1 skipped	100.0%	IN27 658 OUT1 068
✓	NoToolForBestGameCharacterBFCL/irrelevance_206: best game character vs a dragon asked; only a best-WEAPON finder available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 461 OUT114
✓	NoToolForBestMovie2020BFCL/irrelevance_235: best MOVIE of 2020 asked; only a currency-exchange calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 458 OUT94
✓	NoToolForBuildingAPcBFCL/irrelevance_209: how to build a PC asked; only a cooking-recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 453 OUT404
✓	NoToolForFamousPlaceInParisBFCL/irrelevance_210: most famous PLACE in Paris asked; only a restaurant-by-dish search available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 455 OUT43
✓	NoToolForGameShortestPathBFCL/irrelevance_208: shortest in-game path asked; only a taxi-fare calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 463 OUT97
✓	NoToolForInchToMeterBFCL/irrelevance_234: LENGTH conversion (inches to metres) asked; only a CURRENCY converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 454 OUT60
✓	NoToolForLowestGameScoreBFCL/irrelevance_207: LOWEST game score asked; only a HIGHEST-score retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 458 OUT110
✓	NoToolForNewYorkPopulationBFCL/irrelevance_225: current POPULATION of New York asked; only a time-zone converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 456 OUT146
–	NoToolForQuickestFlightLondonTokyoBFCL/irrelevance_236: quickest way to fly London->Tokyo asked; a flight-duration tool is available. REVIEW FLAG (fairness): get_flight_duration plausibly answers this ("quickest ... by plane" ~= quickest flight duration), so a competent human might legitimately call it. Kept as BFCL labels it (irrelevant), but this item is borderline. Marked Hard because the intended "no call" is debatable.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_flight_duration' (with flight_type non-stop) plausibly answers 'quickest way ... by plane' - BFCL-mislabeled/borderline-relevant. Kept for traceability.	Skipped	IN0 OUT0
Tool Calling - Irrelevance: Restaurants & Travel 8 passed	100.0%	IN26 531 OUT1 316
✓	NoToolForBeijingSunsetBFCL/irrelevance_227: SUNSET time asked; only a SUNRISE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 315 OUT56
✓	NoToolForClosestItalianRestaurantBFCL/irrelevance_216: FIND the closest Italian restaurant asked; only a currency converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 314 OUT53
✓	NoToolForFindSushiRestaurantBFCL/irrelevance_214: FIND a sushi restaurant asked; only a tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 316 OUT81
✓	NoToolForNearestPharmacyBFCL/irrelevance_237: FIND the nearest pharmacy asked; only a route-to-a-known-location planner available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 316 OUT54
✓	NoToolForPetFriendlyTrainStationBFCL/irrelevance_230: a pet-friendly TRAIN STATION asked; only hotel search tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 315 OUT177
✓	NoToolForSushiRestaurantsTokyoBFCL/irrelevance_229: FIND sushi RESTAURANTS asked; only hotel-booking tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 316 OUT265
✓	NoToolForThaiRestaurantChicagoBFCL/irrelevance_231: FIND a Thai RESTAURANT asked; only a hotel-availability checker available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 317 OUT74
✓	NoToolForYogaRetreatBFCL/irrelevance_232: a yoga RETREAT asked; only a HOTEL search tool available. REVIEW FLAG (fairness): a retreat is not a hotel, but hotel_search_find_hotels accepts arbitrary amenities (spa, vegetarian) and a location, so it is somewhat tempting. BFCL labels this irrelevant (retreat != hotel); kept as such and marked Hard because the boundary is borderline.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 322 OUT556
Tool Calling - Irrelevance: Sports & Games (set 1) 10 passed	100.0%	IN35 708 OUT1 422
✓	NoToolForAssistAverageBFCL/irrelevance_185: assist average asked; only an average-SCORING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 572 OUT58
✓	NoToolForBestChessMoveBFCL/irrelevance_194: best next move asked; only a board-game-STATISTICS tool is available (the move-suggesting board_analyser lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 574 OUT93
✓	NoToolForChessArrangementCountBFCL/irrelevance_196: total theoretical arrangements asked; only a final-score calculator is available (possible_moves lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 574 OUT302
✓	NoToolForFootballTeamRankingBFCL/irrelevance_186: ranking of a team asked; only a season-MVP tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 570 OUT159
✓	NoToolForIceCreamStoreBFCL/irrelevance_201: find a physical store asked; only a video-game guide is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 567 OUT38
✓	NoToolForMostHeartsInPokerBFCL/irrelevance_199: who holds most of a suit asked; only a poker-dealing tool is available (get_highest_card_holder lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 574 OUT123
✓	NoToolForPlayerBiographyBFCL/irrelevance_189: a biographical "who is X" asked; only a top-ranked-athlete lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 567 OUT172
✓	NoToolForTeamRosterBFCL/irrelevance_192: full roster asked; only a top-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 569 OUT109
✓	NoToolForUnoRulesBFCL/irrelevance_198: the rules of a game asked; only a deck-shuffling tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 571 OUT299
✓	NoToolForWorldCupWinnerBFCL/irrelevance_191: who won a tournament asked (no team named); only a per-team match-stats tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 570 OUT69
Tool Calling - Irrelevance: Sports & Games (set 2) 8 passed1 failed	88.9%	IN34 809 OUT979
✓	NoToolForBlackjackAceRuleBFCL/irrelevance_200: a rules question asked; only a poker highest-card-holder tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 469 OUT101
✓	NoToolForBookAuthorBFCL/irrelevance_205: author of a book asked; only a video-game score calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 473 OUT19
✓	NoToolForChessTournamentWinnerBFCL/irrelevance_195: who won a tournament asked; only a board-position move-suggester is available (the game-history fetch_game_stats lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 470 OUT111
✓	NoToolForCreateJigsawBFCL/irrelevance_204: CREATE a puzzle asked; only a puzzle-SOLVING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 463 OUT157
×	NoToolForHaloPlayerRankBFCL/irrelevance_203: a player's RANK asked; only a player-SCORE lookup is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN7 066 OUT340
✓	NoToolForLakersTopScorerBFCL/irrelevance_193: top scorer of a team asked; only a team-details tool (roster/results/upcoming, not scorers) is available (sports_team.get_top_scorer lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 468 OUT82
✓	NoToolForMonopolyWinnerBFCL/irrelevance_197: who won a casual game asked; only a possible-moves calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 468 OUT56
✓	NoToolForTopTennisPlayerBFCL/irrelevance_190: top ranked PLAYER asked; only a TEAM standing/ranking tool is available (the athlete-ranking get_top_ranked lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 467 OUT63
✓	NoToolForWorldSeriesWinnerBFCL/irrelevance_202: who won a real game asked; only a score calculator (from raw points) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 465 OUT50
Tool Calling - Irrelevance: Sports Statistics 8 passed1 failed	88.9%	IN35 128 OUT1 220
✓	NoToolForBasketballSeasonMvpBFCL/irrelevance_187: who won the MVP asked; only a team-ranking tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 511 OUT41
✓	NoToolForClippersRosterBFCL/irrelevance_179: roster of a team asked; only a two-team game-results tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 504 OUT179
✓	NoToolForCommonChordsInPopBFCL/irrelevance_174: most common chords in a pop song asked; only a key-signature primary-triads tool is available (find_common_chords lives in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 509 OUT396
×	NoToolForCricketMatchParticipantsBFCL/irrelevance_180: who is playing in cricket matches today asked; only a cricket-schedule tool (dates/matches, not participants) is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN7 058 OUT201
✓	NoToolForLaLigaPlayersBFCL/irrelevance_181: who played in a whole league asked; only a two-named-team last-match-result tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 504 OUT79
✓	NoToolForMoonlightSonataComposerBFCL/irrelevance_175: composer of Moonlight Sonata asked; only a blues-scale generator is available here (all composer-lookup tools are isolated in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 507 OUT59
✓	NoToolForNbaMvpWinnerBFCL/irrelevance_184: who won an MVP award asked (no player named); only a per-player stats tool that requires a player name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 513 OUT52
✓	NoToolForTopAssistPlayerBFCL/irrelevance_178: top ASSIST player asked; only a top-GOAL-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 508 OUT156
✓	NoToolForWimbledonChampionBFCL/irrelevance_183: winner of a tournament asked (no person named); only a celebrity-info tool that requires a name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 514 OUT57
Tool Calling - Irrelevance: Time, Weather & Misc 3 passed1 skipped	100.0%	IN7 742 OUT269
✓	NoToolForCityDistanceKmBFCL/irrelevance_239: DISTANCE (km) between two cities asked; only a get_date tool that returns the TIME difference between two locations is available (its 'unit' arg mentions distance, but it computes time) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 584 OUT64
–	NoToolForCurrentTimeSydneyBFCL/irrelevance_228: current time in Sydney asked; a local-time-by-city tool is available. REVIEW FLAG (fairness): get_local_time(location='Sydney') directly answers this, so a competent human would legitimately call it. BFCL nonetheless files it under irrelevance; kept as such to preserve the source label, but this item is effectively RELEVANT and should be reviewed. Marked Hard because the "no call" expectation is contradicted by the tool itself.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_local_time' directly answers 'current time in Sydney' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForTomatoCaloriesBFCL/irrelevance_222: CALORIES in a tomato asked; only a grocery-item-details lookup available. REVIEW FLAG (fairness): item_details at 'detailed' level could arguably surface nutrition info, and it requires a store_location the user never gave; BFCL labels it irrelevant (it is a grocery-store item lookup, not a nutrition-facts service). Kept as such; marked Medium as borderline.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 580 OUT61
✓	NoToolForWriteABookBFCL/irrelevance_217: "can you write a book" asked; only a recipe-CREATION tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN2 578 OUT144
Tool Calling - Irrelevance: Weather, Environment & Services 10 passed	100.0%	IN36 637 OUT1 843
✓	NoToolForBestCo2ReductionBFCL/irrelevance_108: the best way to reduce CO2 asked; only a numeric emission-reduction estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 662 OUT318
✓	NoToolForBestMonthToVisitBFCL/irrelevance_101: best month to visit Hawaii asked; only an average-monthly-temperature tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 660 OUT256
✓	NoToolForCactusNutrientsBFCL/irrelevance_109: a plant's nutrient needs asked; only a weekly watering-needs calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 667 OUT320
✓	NoToolForCurrentTimeWeatherBFCL/irrelevance_103: the current time asked; only a current-weather forecast tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 661 OUT46
✓	NoToolForGardenSpeciesCompositionBFCL/irrelevance_107: garden species composition asked; only a soil sand/silt/clay composition analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 664 OUT144
✓	NoToolForGmoYogaMatBFCL/irrelevance_111: a GMO yoga mat asked; only an eco-friendly product locator is available (GMO is not eco-friendly) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 663 OUT33
✓	NoToolForImpactfulCasesBFCL/irrelevance_94: a firm's impactful cases asked; only a single-case detail lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 671 OUT208
✓	NoToolForPaintingEiffelCaseResultBFCL/irrelevance_96: how long to paint the Eiffel Tower asked; only a law-case result lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 663 OUT92
✓	NoToolForPollinationFlowersBFCL/irrelevance_105: top pollination flower species asked; only a route elevation-profile plotter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 665 OUT221
✓	NoToolForTomatoFertilizerBFCL/irrelevance_106: best fertilizer for tomatoes asked; only a soil-type analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 661 OUT205
Tool Calling - Parallel Calls: Art, Music & Local Services 9 passed1 failed1 skipped	90.0%	IN80 178 OUT3 447
–	CalculateDrivingDistancesShortestRouteBFCL/parallel_38: two calc_distance calls — New York→Washington DC and Los Angeles→San Francisco. Tightened per methodology: the prompt explicitly enables "shortest route", so shortest_route == true is required (schema default is false).0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Skipped: Ambiguous scope (item flaw, not model behavior): the trailing 'with optional parameter shortest route enabled' can reasonably apply to only the second leg (its nearest clause) or to both; the BFCL ground truth requires it on both, but a competent human would not answer deterministically. Kept for traceability.	Skipped	IN0 OUT0
✓	CheckInstrumentAvailabilityInTwoCitiesBFCL/parallel_42: two check_instrument_availability calls — Yamaha P125 in Berlin and Madrid.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 490 OUT237
✓	FindMusicStoresInTwoCitiesBFCL/parallel_41: two find_music_instrument_store calls — San Francisco and Chicago; instruments {Yamaha acoustic guitar, Kawai piano} in any order and either casing.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 515 OUT317
✓	FindRockAndJazzConcertsInTwoCitiesBFCL/parallel_43: four concert_finder calls — {San Francisco, New York} x {rock, jazz} over the next month. time_period may be omitted (defaults to 30 days ≈ "next month"). Isolated from concert_find_nearby (parallel_44) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 810 OUT456
✓	GetExhibitionDetailsForTwoMuseumsBFCL/parallel_40: two museum_exhibition_detail calls — 'Wonder of Nature' at the Louvre and 'Age of Reptiles' at the British Museum. visitor_type {child, adult} (unordered). Tightened per methodology: the prompt explicitly asks for cost "for children and adult", so both visitor types are required (schema default is ['adult'] only).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 552 OUT377
✓	GetMostPlayedPopAndRockSongsBFCL/parallel_45: two musicCharts_getMostPlayed calls — most played Pop and Rock song in Australia. Optional duration omitted (ground truth accepts the schema default of 0).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 474 OUT147
✓	GetMuseumInfoForTwoCitiesBFCL/parallel_39: two museum_info_get_info calls — Washington D.C. and Paris; details {Opening hours, Adult tickets, Child tickets} in any order.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 504 OUT351
×	GetRgbAndHexForThreeColorsBFCL/parallel_37: three color_converter_get_color_info calls — navy, purple, maroon; each with {RGB, HEX} (unordered).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN12 064 OUT508
✓	GetTeamRankingsInTwoLeaguesBFCL/parallel_47: two get_team_ranking calls — Barcelona in UEFA Champions League and Manchester United in La Liga.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 475 OUT296
✓	MixPaintColorsForTwoRequestsBFCL/parallel_36: two paint_color_mixture calls — watercolor magenta and acrylic navy.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 524 OUT319
✓	WinningPercentageForTwoTeamsTwoSeasonsBFCL/parallel_46: four calculate_winning_percentage calls — {Lakers, Bulls} x {2018, 2020}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 770 OUT439
Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 1) 12 passed1 failed	92.3%	IN116 714 OUT5 091
✓	AverageFourNumberSetsBFCL/parallel_132: four calculate_average calls — one per list of numbers (element order preserved).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 768 OUT362
✓	CalculateBmiFourPeopleMetricBFCL/parallel_134: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt states "All measurements are in the metric system", so unit == "metric" is required (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 794 OUT455
×	CalculateCityDistancesInKilometersBFCL/parallel_135: three geo_distance_calculate calls — NY->LA->Miami->NY, in kilometers. Tightened: the tool defaults to miles but the prompt asks "in kilometers", so units == "kilometers" is REQUIRED (omission would return miles, contradicting the request — the ground truth's "omit" option is unsafe here).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN13 451 OUT679
✓	CalculateGpsDistancesAlongRouteBFCL/parallel_133: three calculate_distance calls over GPS coordinate pairs (Paris->Rome->Athens-> Giza), each in kilometers. Uses the coordinate-based tool (coord1/coord2), not the city-name distance tools that are also visible.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 791 OUT426
✓	CompareAverageTemperatureNyAndLaBFCL/parallel_128: two average_temperature calls — New York 7 days in Fahrenheit, Los Angeles 7 days in Celsius. Tightened: the prompt explicitly requests Fahrenheit for NY, so temp_unit == "Fahrenheit" is required (also the default); Celsius is non-default and required for LA.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 362 OUT198
✓	CreateTwoHistogramsWithFiveBinsBFCL/parallel_129: two create_histogram calls — one per data set, 5 bins each (data order preserved).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 826 OUT521
✓	EstimateElephantPopulationThreeScenariosBFCL/parallel_125: three elephant_population_estimate calls — current 500 at 2%, 1.5%, 2.5% over 10y.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 703 OUT375
✓	FindItalianRestaurantsInNyAndLaBFCL/parallel_130: two find_restaurants calls — New York and Los Angeles, Italian, 4 results each, dietary_requirements [vegan, gluten-free]. Distinct from restaurant_find_nearby (this tool carries food_type + number, which the prompt supplies, so it is the correct choice here).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 426 OUT282
✓	FindRestaurantsByDietInThreeCitiesBFCL/parallel_127: three restaurant_find_nearby calls — New York, Los Angeles, Chicago, each with dietary_preference [Vegan, Gluten-free, Dairy-free]. List order follows the prompt (matches ground truth).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 549 OUT328
✓	PredictEvolutionDarwinAndLamarckBFCL/parallel_126: two prediction_evolution calls — African Elephant over 5000y, Darwin then Lamarck. Tightened per methodology: the prompt explicitly says "using the Darwin model", so model == "Darwin" is required for the first call (ground truth also accepts omission via the 'Darwin' default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 385 OUT275
✓	ProjectPopulationGrowthJapanAndIndiaBFCL/parallel_124: four population_projections calls — Japan/10y and India/20y, each at the current (default) growth rate and at an explicit rate. "current growth rate" maps to the default (growth_rate omitted, accepts null or 0.01); the explicit 1.5%/2.1% rates are required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 710 OUT465
✓	RouteThreeLegsAvoidingTollsBFCL/parallel_131: three map_routing_fastest_route calls — SF->Palo Alto->San Jose->home, all avoiding tolls. Tightened: the prompt explicitly says "avoiding toll roads", so avoid_tolls == true is required for every leg (non-default value stated in the prompt).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 555 OUT284
✓	ShortestCityDistanceBusWithAndWithoutTransferBFCL/parallel_136: two city_distance_find_shortest calls — NY->LA by bus, first without transfer then allowing transfer. Tightened: the prompt explicitly says "using a bus", so transportation == "bus" is required for both (also the default). allow_transfer: false/omitted for the first, true for the second.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 394 OUT441
Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 2) 14 passed	100.0%	IN125 846 OUT4 659
✓	CalculateAreaOfThreeTrianglesBFCL/parallel_143: three calc_area_triangle calls — (10,5), (15,7), (20,10).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 962 OUT298
✓	CalculateBinomialProbabilityBasketballBFCL/parallel_161: three calculate_binomial_probability calls — basketball shooter at 60%: (10,7), (15,10), (20,15). probability_of_success == 0.6 is required (prompt states 60%). ISOLATED here (away from prob_dist.binomial in ToolCalling-Parallel-D3): both tools compute binomial probability, and this prompt does not name a function, so having both visible would make the choice genuinely ambiguous. Kept apart so exactly one correct tool is available.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 041 OUT375
✓	CalculateBmiInMetersForThreePeopleBFCL/parallel_138: three calculate_BMI calls (weight kg, height meters) — John, Sarah, Mike.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 023 OUT341
✓	CalculateCelestialDistancesInMilesBFCL/parallel_149: two calculate_distance calls between celestial bodies (Mars-Venus, Mars-Jupiter), in miles. Uses the body-name distance tool (body1/body2). Tightened: prompt asks "in miles", so unit == "miles" is required (tool default is kilometers, so omission would be wrong).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 768 OUT249
✓	CalculateClockAnglesAtThreeTimesBFCL/parallel_145: three calculate_clock_angle calls — 3:15, 8:20, 11:50. Tightened: the prompt asks for results "rounded to 2 decimal places", so round_to == 2 is required (also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 943 OUT359
✓	CalculateFactorialOfFourNumbersBFCL/parallel_144: four math_factorial calls — 5, 3, 4, 2.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 972 OUT237
✓	CalculateLightTravelTimeThreeLegsBFCL/parallel_147: three light_travel_time calls — 4.22, 6.1, 5.88 light years. Tightened: the prompt explicitly states the speed of light (299792458 m/s), so speed_of_light == 299792458 is required (also default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 029 OUT385
✓	CalculateSpeedForCarBicycleTrainBFCL/parallel_148: three calculate_speed calls — car 500m/25s in km/h, bicycle 1000m/200s in m/s, train 10000m/600s in km/h. Each output unit is explicitly named in the prompt, so to_unit is required.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 044 OUT305
✓	FetchAllDataFieldsForTwoEmployeesBFCL/parallel_139: two employee_fetch_data calls — Tech Solutions, employees 12345 and 67890, each fetching all four data fields (order follows the prompt / ground truth).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 876 OUT270
✓	FindDiCaprioDramaAndComedyMoviesBFCL/parallel_140: two imdb_find_movies_by_actor calls — Leonardo DiCaprio, Drama in 2010 and Comedy in 2012. Tightened: the prompt explicitly names each category, so category is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 764 OUT244
✓	GetImaxAndTwoDTheaterReleasesBFCL/parallel_141: two get_theater_movie_releases calls — New York IMAX over 7 days, Los Angeles 2D over 14 days. Tightened: the prompt explicitly names each format, so format is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 793 OUT268
✓	PlotTwoSineWavesBFCL/parallel_146: two plot_sine_wave calls with all five parameters supplied by the prompt.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 899 OUT295
✓	SortFourListsAscendingAndDescendingBFCL/parallel_137: eight array_sort calls — four lists, each sorted ascending AND descending (list element order preserved as given).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 848 OUT772
✓	UpdateTwoCustomersNameAndEmailBFCL/parallel_142: two update_user_info calls — users 12345 and 67890, both set name "John" and email "example@.com". update_info is a nested object. Tightened: the prompt names the 'CustomerInfo' database, so database == "CustomerInfo" is required (also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 884 OUT261
Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 3) 11 passed	100.0%	IN87 979 OUT4 204
✓	AreaUnderCurveForTwoPolynomialsBFCL/parallel_150: two mathematics_calculate_area_under_curve calls — polynomial [3,-2,1] over [-1,2] and [1,0,-1] over [0,3] (coefficient/limit order preserved).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 834 OUT246
✓	BinomialProbabilitiesThreeGamesBFCL/parallel_160: three prob_dist_binomial calls — (20,5), (50,15), (100,30), each with p == 0.3 (prompt states a 0.3 win probability). The prompt explicitly names 'prob_dist.binomial'; the colliding calculate_binomial_probability tool is kept in a separate script (ToolCalling-Parallel-D2).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 005 OUT656
✓	CalculateBmiSystemFourPeopleBFCL/parallel_154: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt says "use the metric system for all calculations", so system == "metric" is required (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 132 OUT459
✓	DiceRollProbabilitiesThreeScenariosBFCL/parallel_159: three probability_dice_roll calls — roll 4 three times and roll 2 twice on a six-sided die (die_sides default 6, accepts omission), then roll 7 twice on an eight-sided die (die_sides == 8 required, non-default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 962 OUT396
✓	GenerateNormalVariatesTwoConfigsBFCL/parallel_158: four random_normalvariate calls — two at (mu 5, sigma 2) and two at (mu 10, sigma 3). Duplicate calls: each configuration must appear exactly twice (multiplicity 2).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 068 OUT357
✓	PredictPricesForFourHousesBFCL/parallel_157: four predict_house_price calls — New York (3/2/1500), Los Angeles (4/3/2000), Chicago (2/1/1200), Miami (3/2/1800).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 256 OUT510
✓	RaiseTwoNumbersToPowersBFCL/parallel_152: two math_power calls — 3^5 and 2^3, no modulus (mod omitted / null).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 688 OUT136
✓	RunLinearRegressionWithAndWithoutStandardizationBFCL/parallel_155: two run_linear_regression calls — same predictors [Age, Income, Education] and target "Spending Score", first without standardization then with. standardize is explicitly requested (false then true), so it is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 880 OUT265
✓	TrainRandomForestFourConfigsBFCL/parallel_156: four random_forest_train calls on "data_random_forest" — (100,10), (200,20), (300,30), (400,40). The prompt explicitly names this function, disambiguating it from train_random_forest_classifier visible in this script.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 314 OUT519
✓	TrainRfClassifierTwoConfigsBFCL/parallel_153: two train_random_forest_classifier calls — dataset1 (depth 10, 100 trees) and dataset2 (depth 20, 200 trees). The prompt explicitly names this function, disambiguating it from the other random-forest training tool (random_forest_train) visible in this script.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 878 OUT260
✓	TriangleAreasInDifferentUnitsBFCL/parallel_151: three geometry_area_triangle calls in different units — (15,20) square meters, (25,30) square feet, (35,40) square inches. First unit is the default (accepts omission / square meters); feet and inches are non-default and required.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 962 OUT400
Tool Calling - Parallel Calls: Finance, Stats & Legal 13 passed	100.0%	IN110 804 OUT3 922
✓	BuildTwoDecisionTreeClassifiersBFCL/parallel_12: two model_DecisionTreeClassifier calls — gini/depth 5 and entropy/depth 10, both random_state 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 444 OUT234
✓	CalculateFutureValueForThreeTermsBFCL/parallel_18: three financials_calculate_future_value calls — years 1, 5, 10 (same 1000 @ 0.05).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 606 OUT337
✓	CalculatePresentValueForThreeTermsBFCL/parallel_14: three calculate_present_value calls — years 10, 20, 30 (same payment 1000, rate 0.05).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 603 OUT295
✓	CapitalGainsTaxForTwoStatesBFCL/parallel_15: two calculate_capital_gains_tax calls — CA (15000/25000) and FL (20000/50000).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 496 OUT303
✓	ComputePythagorasForTwoTrianglesBFCL/parallel_10: two math_pythagoras calls — sides (3,4) and (5,12).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 387 OUT192
✓	ConfidenceIntervalForTwoSampleSizesBFCL/parallel_13: two confidence_interval_calculate calls — sample sizes 50 and 150. Tightened per methodology: the prompt explicitly requests "95% confidence interval", so confidence_level == 0.95 is required (schema default is 0.9, but the prompt states the value).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 499 OUT475
✓	CourtCaseStatusAndTrialDatesBFCL/parallel_22: four court_info_get_case_status calls — cases XY1234 (LA) and GH5678 (Orange County), each queried for status (details may be omitted via the 'status' default) and for trial_date.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 758 OUT369
✓	GetStockPriceAndVolumeForThreeCompaniesBFCL/parallel_17: three get_stock_data calls — AAPL, GOOG/GOOGL, MSFT; each with data_points {price, volume} (unordered).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 575 OUT435
✓	LoanEligibilityForTwoInstitutionsBFCL/parallel_20: two loan_eligibility_check calls — HSBC (500k/100k) and Wells Fargo (700k/120k).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 511 OUT250
✓	MortgagePaymentForThreeLoanTermsBFCL/parallel_19: three calculate_mortgage_payment calls — loan terms 15, 20, 30 (400000 @ 0.04).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 631 OUT307
✓	PredictHousePriceInTwoCitiesBFCL/parallel_11: two ml_predict_house_price calls — New York 3000 sq ft, Los Angeles 4000 sq ft.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 418 OUT245
✓	ReturnOnInvestmentGainAndLossBFCL/parallel_16: two calculate_return_on_investment calls — gain 500 on 2000, loss -1000 on 5000.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 449 OUT202
✓	SearchMoneyLaunderingConvictionsBFCL/parallel_21: two law_crimes_search calls — money laundering in San Francisco 2019 and Texas 2018.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 427 OUT278
Tool Calling - Parallel Calls: Legal, History & Environment 12 passed1 failed1 skipped	92.3%	IN123 287 OUT4 780
✓	BookTwoConnectingFlightsBFCL/parallel_30: two book_flight calls — San Francisco→Tokyo (2022-05-03) and Tokyo→Sydney (2022-05-18).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 061 OUT212
✓	CalculateAlimonyForTwoDurationsBFCL/parallel_23: two alimony_calculator_ca_calculate calls — durations 10 and 20 years (10000/3000 income).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 124 OUT431
✓	CalculateWasteForFamilyAndBachelorBFCL/parallel_29: two waste_calculation_calculate calls — LA family (2 adults, 2 children) and New York bachelor (1 single). The population argument is a nested object.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 074 OUT245
✓	FetchTwoHistoricalFactsBFCL/parallel_31: two history_fact_fetch calls — Treaty of Paris and Magna Carta. Optional depth/year omitted (ground truth accepts the schema defaults).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 980 OUT414
✓	FindClassicalConcertsWithParkingBFCL/parallel_44: two concert_find_nearby calls — classical concerts near Berlin and Paris next Friday. Tightened per methodology: the prompt explicitly asks for concerts "with available parking", so amenities == ['Parking'] is required (this also matches the schema default, but the prompt states it). Isolated from concert_finder (parallel_43) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 038 OUT195
–	FindCourtCasesByNumberAndTypeBFCL/parallel_26: four court_case_find calls — the cross product of case numbers {67813, 71249} and types {Civil, Criminal}; case_number is a single-element array; Civil may be omitted via its default. FLAGGED as borderline (see report): the prompt "case numbers '67813','71249' for type 'Civil' and 'Criminal'" is ambiguous about the pairing; BFCL ground truth expects all four combinations, kept faithful.0 AI queries 0 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL Skipped: Ambiguous call decomposition (item flaw, not model behavior): 'case numbers 67813, 71249 for type Civil and Criminal' has no single deterministic reading - two calls (both numbers per type, case_number as a list) is as valid as the BFCL 2x2 = four single-number calls; a competent human would not answer deterministically. Kept for traceability.	Skipped	IN0 OUT0
✓	FindNatureReservesNearTwoCitiesBFCL/parallel_27: two nature_reserve_find_nearby calls — Berkeley (10km, picnic tables + restrooms) and Tokyo (5km, playgrounds + biking trails). Amenity list order is unordered.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 062 OUT225
✓	GetCaseDetailsWithHistoryAndLitigantsBFCL/parallel_24: two law_case_get_details calls — cases 28473 and 64725. Tightened per methodology: the prompt explicitly asks for "their history and details of litigants", so include_history == true and include_litigants == true are required (schema defaults are false).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 044 OUT282
✓	GetEventsForTwoPresidenciesBFCL/parallel_32: two us_history_events_by_presidency calls — Abraham Lincoln and George Washington. Optional start_year/end_year omitted (ground truth accepts the schema defaults).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 975 OUT538
×	GetPresidentsAndVicePresidentsByYearBFCL/parallel_33: four get_president_and_vp calls — president for 1980 and 2016, vice president for 1975 and 2011.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN14 837 OUT620
✓	GetReligionInfoForTwoEmpiresBFCL/parallel_35: two ancient_empires_get_religion_info calls — Persian Empire and Mauryan Empire. Tightened per methodology: the prompt explicitly asks for "religious history and influences", so include_influences == true is required (schema default is false).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 015 OUT263
✓	GetTemperatureForTwoCitiesBFCL/parallel_28: two get_current_and_future_temperature calls — Seattle and Los Angeles, next 3 hours.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 987 OUT170
✓	LookupLawsuitsForTwoYearsBFCL/parallel_25: two lawsuit_lookup calls — Dara Inc for years 2019 and 2018.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 032 OUT165
✓	TrackChristianityInTwoRegionsBFCL/parallel_34: two religion_history_track calls — Christianity in Egypt and Turkey, 100–1500 A.D.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 058 OUT1 020
Tool Calling - Parallel Calls: Markets & Environment 13 passed	100.0%	IN114 815 OUT5 328
✓	CalculateCagrForTwoInvestmentsBFCL/parallel_176: two calculate_cagr calls — (5000,7000,5) and (8000,12000,3).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 669 OUT310
✓	CalculateEmissionSavingsForSolarAndWindBFCL/parallel_186: two calculate_emission_savings calls — solar(12 months) and wind(8 months), Midwest region. Prompt states the Midwest region explicitly -> region required (accept "Midwest"/"Midwest region").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 562 OUT294
✓	CompareFutureValueWithAndWithoutContributionsBFCL/parallel_182: two finance_calculate_future_value calls — (5000,0.07,10, no contribution) and (3000,0.06,10, contribution 200). First scenario states "no additional contributions" -> null or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 729 OUT253
✓	ComputeEuclideanNormForThreePointsBFCL/parallel_183: three math_hypot calls — (5,7), (10,15), (20,25). z omitted or 0 (2D points).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 765 OUT383
✓	EstimatePopulationsForTwoSpeciesAcrossTwoYearsBFCL/parallel_185: four estimate_population calls — Bengal Tigers/India and African Elephants/Kenya for 2021 and for 2023. NOTE: upstream ground truth's third entry lists year "" (omitted -> default 2020), which is an upstream error: the prompt explicitly asks for 2023 for "both these species". We tighten both 2023 estimates to year == 2023 (fair deterministic answer).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 908 OUT627
✓	FindRootsForTwoQuadraticEquationsBFCL/parallel_184: two algebra_quadratic_roots calls — (3,7,2) and (5,-4,1).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 682 OUT340
✓	GetClosingPricesForTwoCompaniesOnTwoDatesBFCL/parallel_178: four get_stock_price (by-date variant) calls — the 2x2 cross product of {Microsoft, Apple} x {2022-01-01, 2022-02-01} on NASDAQ. Upstream ground truth's first entry loosely lists company_name as [Microsoft, Apple]; the deterministic human answer is the four distinct (company, date) combinations, which we require. exchange NASDAQ (default) may be omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 940 OUT599
✓	GetDetailedAirQualityForThreeDaysBFCL/parallel_187: three get_air_quality calls — New York City, detail true, for 2023-05-05, 05-04, 05-03. Prompt asks for PM2.5/PM10/ozone/sources -> detail == true; "today/yesterday/day before" with today = May 5, 2023.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 768 OUT370
✓	GetFourPriceTypesForTwoCompaniesBFCL/parallel_180: eight stock_price calls — {Microsoft, Apple} x {Open, Close, High, Low}, days 30. Prompt explicitly lists all four data types, so data_type must be set explicitly (required) to distinguish the eight calls; omission (default 'Close') cannot represent the four requested types.1 AI query 8 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL	Passed	IN9 490 OUT775
✓	GetPricePerOunceForFourMetalsBFCL/parallel_177: four get_metal_price calls — gold, silver, platinum, palladium, measure 'ounce'. Prompt asks "price per ounce" -> measure == "ounce" required.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 789 OUT340
✓	GetStockPricesForFourCompaniesAndDurationsBFCL/parallel_181: four get_stock_prices calls — Apple(1 week), Microsoft(2 weeks), Amazon(3 weeks), Tesla(1 month). Each 'companies' argument is a single-element list.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 838 OUT282
✓	GetTrafficInfoForThreeLegsWithModesBFCL/parallel_188: three get_traffic_info calls — NY->LA (driving), LA->SF (bicycling), SF->NY (transit). First leg explicitly says "driving" -> mode == "driving" required (tightened; also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 763 OUT301
✓	ProjectFutureValueForThreeStocksBFCL/parallel_175: three portfolio_future_value calls — AAPL(5000,0.07,5), MSFT(8000,0.06,7), AMZN(10000,0.08,10).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 912 OUT454
Tool Calling - Parallel Calls: Mathematics & Physics (set 1) 13 passed	100.0%	IN104 090 OUT3 829
✓	AreaOfTwoRightTrianglesInMetersBFCL/parallel_68: two calculate_area calls — right triangles base/height 12x15 and 18x24 in meters. unit required (tool default is 'cm'; the prompt states meters/square meters).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 994 OUT281
✓	CheckPlayerStatusAcrossThreeSeasonsBFCL/parallel_49: three player_status_check calls — player 3142 in RocketLeague for seasons 2017/2018/2019. Seasons are explicitly given, so each is required (default 2023 not applicable).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 075 OUT256
✓	CircumferenceOfFourCirclesInCmBFCL/parallel_65: four calculate_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt gives every radius in cm (e.g. "5cm") and the tool default unit is 'm', so a correct call must pass unit == 'cm'/'centimeter' (ground truth requires it on the first call and merely allows omission on the rest; we require it consistently since it is stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 217 OUT401
✓	ConvertEuroAndPoundsToDollarsBFCL/parallel_59: two get_conversion_cost calls — 15000 Euro->dollars and 200 pounds->dollars.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 907 OUT303
✓	FinalVelocityForFourDropsBFCL/parallel_81: four calculate_final_velocity calls — heights 10/20/15/25, initial_velocity 0 (stated), gravity 9.81 (default, may be omitted).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 402 OUT440
✓	FindItalianAndJapaneseRestaurantsBFCL/parallel_52: two restaurant_finder calls — Italian in New York (preferences ['Vegetarian']) and Japanese in Los Angeles (delivery; ['Delivery'] or omitted since default is ['Delivery']).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 888 OUT220
✓	FindRootsOfTwoQuadraticsAlgebraBFCL/parallel_62: two algebra_quadratic_roots calls — 3x^2+4x+2 and 5x^2-7x+3.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 054 OUT272
✓	FindSoupAndVeganSaladRecipesBFCL/parallel_51: two recipe_search_find calls — Chicken Noodle Soup (diet omitted/default 'Keto') and a Vegan Salad (diet == 'Vegan', dish name accepts several phrasings).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 835 OUT174
✓	GcdOfJohnAndMaryNumbersBFCL/parallel_80: two math_gcd calls — gcd(36,48) and gcd(60,96). Same tool as parallel_76.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 950 OUT300
✓	GcdOfTwoPairsBFCL/parallel_76: two math_gcd calls — gcd(45,60) and gcd(81,27).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 882 OUT154
✓	GetPokemonMovesAndCheckSolarBeamBFCL/parallel_48: two PokemonGO_get_moves calls — Pikachu's move list (move omitted/default 'Run') and a check whether Bulbasaur can learn 'Solar Beam'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 872 OUT296
✓	PrimeFactorsFormattedAndArrayBFCL/parallel_74: four get_prime_factors calls — numbers 4567 and 7890, each formatted true and false (user wants "a formatted string as well as an array").1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 164 OUT598
✓	SaveGameProgressEasyAndHardBFCL/parallel_50: two game_save_progress calls — stage 7 easy and stage 3 hard; level omitted/default 'user'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 850 OUT134
Tool Calling - Parallel Calls: Mathematics & Physics (set 2) 13 passed	100.0%	IN107 973 OUT4 846
✓	AreaOfThreeCirclesInMetersBFCL/parallel_66: three geometry_area_circle calls — radii 5/10/15 in meters (default is meters, so units may be omitted).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 339 OUT379
✓	BookTwoHotelRoomsBFCL/parallel_57: two hotel_booking_book calls — Sheraton (NY, 2 adults/1 child) and Marriott (LA, 1 adult/2 children).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 387 OUT364
✓	CheckAvailabilityForTwoHotelsBFCL/parallel_56: two hotel_booking_check_availability calls — Queens Hotel (Berlin) and Royal Hotel (Paris), each with check-in/out dates (several date formats accepted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 319 OUT295
✓	CircumferenceOfFourCirclesInMetersBFCL/parallel_70: four geometry_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt asks for output in meters and gives radii in m, but the tool default unit is 'cm', so a correct call must pass units == 'm'/'meters' (ground truth requires it on the first call and only allows omission on the rest; we require it consistently since it is explicitly stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 519 OUT471
✓	DerivativeOfPolynomialAndItsDerivativeBFCL/parallel_71: two calculate_derivative calls — 3x^3-2x^2+5x-7 at x=4, then its derivative 9x^2-4x+5 at x=2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 234 OUT377
✓	FinalVelocityForThreeDragRaceRoundsBFCL/parallel_83: three final_velocity calls — (v0=0,a=5,t=10), (v0=10,a=7,t=8), (v0=20,a=4,t=12).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 516 OUT465
✓	FindGroceryStoresByCriteriaBFCL/parallel_55: two grocery_store_find_by_criteria calls — NYC ['24 hours'] and San Diego ['Home Delivery'].1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 152 OUT147
✓	GetTwoCookingRecipesBFCL/parallel_53: two get_cooking_recipe calls — Lasagne Bolognese for 4 and Caesar Salad for 2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 170 OUT298
✓	GetTwoExchangeRatesBFCL/parallel_58: two get_exchange_rate calls — USD->AUD and USD->CAD.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 134 OUT221
✓	HighestCommonFactorOfFourPairsBFCL/parallel_77: four math_hcf calls — (45,60), (90,120), (36,48), (72,96). FLAGGED (see final report): the prompt is phrased as chaining ("use that result to find the HCF with...") but the ground truth treats it as four INDEPENDENT parallel calls on the literal pairs; a human might interpret it as sequential composition. Kept BFCL-faithful (four independent literal-pair calls) since parallel semantics preclude feeding one call's result into another.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 553 OUT1 014
✓	OrderSameItemsFromTwoWholeFoodsLocationsBFCL/parallel_54: two whole_foods_order calls — same two items (order-independent) at downtown and uptown, size large.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 200 OUT215
✓	PrimeFactorizeAsDictionaryBFCL/parallel_79: two prime_factorize calls — 4567 and 7890, return_type 'dictionary' (explicitly requested).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 176 OUT371
✓	SolveTwoQuadraticEquationsBFCL/parallel_63: two solve_quadratic_equation calls — (5,6,1) and (3,2,1). Prompt names this function explicitly.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 274 OUT229
Tool Calling - Parallel Calls: Mathematics & Physics (set 3) 12 passed	100.0%	IN98 303 OUT4 071
✓	AreaOfTwoCirclesInMetersBFCL/parallel_67: two geometry_calculate_area_circle calls — radii 5 and 10 in meters. unit required (tool default is 'cm'; the prompt states meters).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 023 OUT288
✓	AreaOfTwoTrianglesBFCL/parallel_69: two calculate_triangle_area calls — base/height 10x5 and 8x6.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 083 OUT199
✓	DerivativeOfTwoFunctionsAtValuesBFCL/parallel_73: two calculus_derivative calls — 3x^2+2x-1 at 5 (variable x, default) and 4y^3-3y^2+2y-1 at 3 (variable y).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 128 OUT359
✓	DisplacementForThreeCarsBFCL/parallel_84: three calculate_displacement calls — (15,7,accel 3.5), (20,10,accel 2.0), (25,8,no acceleration). The third car travels "without any acceleration" so acceleration == 0 (also the default; omission accepted).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 344 OUT322
✓	EuclideanNormForThreePointsBFCL/parallel_61: three math_hypot calls — (3,4) and (6,8) with z omitted/0, and (9,12,15).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 258 OUT423
✓	FactorialOfThreeNumbersBFCL/parallel_60: three math_factorial calls — 5, 7, 9.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 085 OUT180
✓	FinalSpeedForTwoFreeFallObjectsBFCL/parallel_85: two calculate_final_speed calls — initial_speed 0 for 10 s and initial_speed 5 for 7 s; gravity -9.81 (default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 151 OUT450
✓	GcdViaNumberTheoryBFCL/parallel_78: two number_theory_gcd calls — (45,60) and (81,63). Prompt names this function explicitly.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 026 OUT155
✓	IntegrateTwoFunctionsWithTwoMethodsBFCL/parallel_72: four integrate calls — x3 on [2,5] and 2x2+3x-1 on [-1,3], each with trapezoid (default, may be omitted) and simpson (explicit).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 476 OUT487
✓	PrimeFactorsOfThreeNumbersBFCL/parallel_75: three number_analysis_prime_factors calls — 45, 100, 150.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 091 OUT250
✓	SolveFourQuadraticsWithRootTypeBFCL/parallel_64: four solve_quadratic calls — (2,5,3) all roots, (1,-3,2) real, (4,-7,3) all roots, (1,2,1) real. 'all' is the default so it may be omitted; 'real' is explicitly requested and required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 542 OUT776
✓	VelocityForTwoCyclingDaysBFCL/parallel_82: two calculate_velocity calls — 120 km in 5 h and 150 km in 6 h; unit km/h (default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 096 OUT182
Tool Calling - Parallel Calls: Mixed Domains 9 passed1 failed	90.0%	IN73 573 OUT2 593
✓	CalculateBmiForTwoPeopleBFCL/parallel_4: two calculate_bmi calls — 6 ft / 80 kg and 5.6 ft / 60 kg.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 282 OUT288
✓	CalculateEmForceForTwoTimeIntervalsBFCL/parallel_1: two calculate_em_force calls — same field/area, d_time 4 then 10.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 303 OUT218
✓	CalculateFactorialForThreeNumbersBFCL/parallel_7: three math_factorial calls — 5, 10, 15.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 342 OUT205
✓	CalculateResistanceForCopperAndAluminumBFCL/parallel_2: two calculate_resistance calls — copper then aluminum, same length/area. Tightened per methodology: the prompt explicitly names "resistivity of copper", so the first call must pass resistivity == "copper" (ground truth also accepts omission via the 'copper' default, but an explicitly-stated value must be supplied). Aluminum is non-default → required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 290 OUT373
✓	CalculateSalesTaxForThreeCitiesBFCL/parallel_6: three calculate_sales_tax calls — Chicago/IL $30.45, Sacramento/CA $52.33, Portland/OR $11.23.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 474 OUT290
✓	FetchPopulationForCitiesStateAndCountryBFCL/parallel_8: four database_us_census_get_population calls — NYC (city), LA (city), Alaska (state), USA (country); year defaults to 2000 (may be omitted). Requires area-type inference.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 558 OUT261
×	FindMovieShowingsForTwoMoviesBFCL/parallel_9: two find_movie_showing calls near San Diego — Tenet at 5 pm and No Time To Die at 7:30 pm.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: match0 == 1 Values: match0 = 0	Failed	IN7 296 OUT253
✓	GetSequenceAnd3DForThreeProteinsBFCL/parallel_3: three protein_info_get_sequence_and_3D calls — HbA1c, normal + rat hemoglobin. Tightened per methodology: the prompt explicitly asks for "their 3D models", so model_3d == true is required (ground truth also accepts omission via the default of true).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 421 OUT252
✓	ListShowRatingsOnNetflixAndHuluBFCL/parallel_5: two streaming_services calls — Netflix['Friends'] and Hulu['The Office','Stranger Things'] (Hulu show order unordered), both sorted by rating. Must group shows per service.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 341 OUT313
✓	PlayTwoArtistsForGivenDurationsBFCL/parallel_0: two spotify_play calls — Taylor Swift for 20 min and Maroon 5 for 15 min.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 266 OUT140
Tool Calling - Parallel Calls: Physical & Life Sciences (set 1) 12 passed	100.0%	IN102 090 OUT3 981
✓	ElectricFieldForTwoChargesBFCL/parallel_94: two calculate_electric_field calls — charge 5/distance 2 and charge 3/distance 4. "Conducted in a vacuum" = the default permitivity → may be omitted or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 414 OUT195
✓	ElectromagneticForceInTwoMediaBFCL/parallel_96: two electromagnetic_force calls — same charges 5,10 and distance 2, differing by medium_permittivity. Vacuum call (8.854e-12) equals the default → may be omitted or 8.854e-12; the second medium (5e-12) is non-default → required.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 570 OUT282
✓	EstimateTravelTimeForThreeRoutesBFCL/parallel_93: three route_estimate_time calls — (1) NYC->Miami via Philadelphia/Washington D.C./Atlanta, (2) NYC->Miami skipping Atlanta (via Philadelphia/Washington D.C.), (3) Philadelphia-> Miami via Washington D.C. Stop orderings from ground truth accepted. FLAG parallel_93: BORDERLINE. Ground truth allows many stop-list variants, some including the start city itself as a stop; a competent human's stop set is not fully deterministic. Kept BFCL-faithful by accepting the natural stop orderings (start city excluded from stops).1 AI query 3 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL	Passed	IN8 565 OUT320
✓	FinalSpeedForTwoDroppedObjectsBFCL/parallel_88: two calculate_final_speed calls — v0 0/height 10 and v0 5/height 20. gravity stated as 9.8 (the default) so it may be omitted or 9.8 per ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 420 OUT277
✓	FinalVelocityForTwoCarsBFCL/parallel_87: two calculate_final_velocity calls — Car A (v0 0, a 6, t 10), Car B (v0 20, a 4, t 15).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 437 OUT236
✓	FinalVelocityFromDistanceForTwoObjectsBFCL/parallel_86: two kinematics_final_velocity_from_distance calls — accel 5/dist 100 and accel 10/dist 200. "Both objects start from rest" → initial_velocity may be omitted or 0 (default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 409 OUT343
✓	FindVeganRestaurantsInThreeCitiesBFCL/parallel_91: three vegan_restaurant_find_nearby calls — LA, SF, Seattle, each with operating_hours 22 (prompt: "open until at least 22:00", a non-default value → required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 466 OUT300
✓	GenerateFourTravelItinerariesBFCL/parallel_90: four travel_itinerary_generator calls — Tokyo/7/200/urban, Paris/10/150/history, Sydney/5/100/nature, Rome/12/180/culture. Tokyo's "urban" equals the default → omit or "urban".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 817 OUT580
✓	GetDirectionsForRoundTripBFCL/parallel_89: three get_directions calls — SF->Palo Alto (fastest), Palo Alto->Golden Gate Bridge (scenic), Golden Gate Bridge->SF (fastest). route_type explicitly stated each time.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 516 OUT389
✓	MagneticFieldForTwoLoopsBFCL/parallel_95: two calculate_magnetic_field calls — current 10/radius 0.5 and current 15/radius 1.0. "Same as in free space" = default permeability → may be omitted or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 518 OUT359
✓	ResonantFrequencyForTwoCircuitsBFCL/parallel_97: two calculate_resonant_frequency calls — L 0.005/C 1e-07/round 3 and L 0.007/C 2e-07/round 4. Rounding is stated explicitly → required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 484 OUT328
✓	ShortestDrivingDistanceRoundTripBFCL/parallel_92: three get_shortest_driving_distance calls — NYC->LA, LA->Miami, Miami->NYC, all in miles (prompt explicitly "in miles" → non-default unit required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 474 OUT372
Tool Calling - Parallel Calls: Physical & Life Sciences (set 2) 13 passed	100.0%	IN111 763 OUT5 669
✓	AbsolutePressureAtTwoAltitudesBFCL/parallel_102: two calc_absolute_pressure calls — gauge 2.5 with atm 1 (default; may be omitted) and gauge 2.5 with atm 0.85 (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 523 OUT222
✓	BoilingMeltingPointsAcrossSeaLevelsBFCL/parallel_100: four get_boiling_melting_points calls — water@0, iron@1000, water@1000, iron@0. FLAG parallel_100: BORDERLINE. The prompt ("water and iron at sea levels of 0 and 1000 meters respectively") reads most naturally as TWO calls (water@0, iron@1000), but the BFCL ground truth is the full 2x2 cross product (FOUR calls). Kept BFCL-faithful to the ground truth; a competent human could reasonably answer with two calls, so this item's expected call count is not fully deterministic.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 409 OUT295
✓	CellInfoDetailedAndBriefBFCL/parallel_108: two biology_get_cell_info calls — neuron with detailed=true (prompt: "detailed description"), muscle with detailed=false ("less detailed", the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 412 OUT884
✓	DensityForTwoSubstancesBFCL/parallel_101: two calculate_density calls (mass/volume schema) — 10 kg / 2 m³ and 15 kg / 3 m³. unit "kg/m³" equals the default → may be omitted or "kg/m³".1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 518 OUT194
✓	ElectricFieldStrengthAtSeveralDistancesBFCL/parallel_98: four calculate_electric_field_strength calls — charge 2, distances 0.5/1/2 in vacuum (default medium), plus distance 1 in air. The distance-1 vacuum and distance-1 air calls are distinguished by medium. Vacuum equals the default → may be omitted or "vacuum"; "air" is required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 835 OUT408
✓	EntropyChangeForTwoConditionsBFCL/parallel_103: two entropy_change_calculate calls — substance A, mass 2, temps 25->75 and 10->50. pressure 1 equals the default → may be omitted or 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 646 OUT401
✓	FetchDnaSequenceThreeWaysBFCL/parallel_106: three fetch_DNA_sequence calls for id XYZ123 — default fasta/0, then genbank format, then 500 bp upstream. Defaults (fasta, upstream 0) may be omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 617 OUT327
✓	FunctionLookupForTwoMoleculesBFCL/parallel_110: two cell_biology_function_lookup calls — ATP in mitochondria and DNA in nucleus, both with specific_function=true (prompt asks for a specific function within the organelle).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 473 OUT360
✓	HeatCapacityForThreeConditionsBFCL/parallel_105: three calc_heat_capacity calls — 300K/2.5m³, 350K/2.5m³, 300K/1.5m³, all for air. "air" equals the default gas → may be omitted or "air".1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 694 OUT432
✓	MolecularWeightForTwoCompoundsBFCL/parallel_111: two calculate_molecular_weight calls — Glucose (C6H12O6) and Sucrose (C12H22O11), both in grams/mole (prompt explicitly requests that unit; ground truth also accepts "g/mol").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 474 OUT400
✓	PhaseChangeEnergyForWaterBFCL/parallel_99: four thermo_calculate_energy calls — 500 g water, phase melting/freezing/ vaporization/condensation. Substance "water" equals the default → may be omitted or "water".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 883 OUT701
✓	ProteinSequencesForTwoGenesTwoSpeciesBFCL/parallel_107: four get_protein_sequence calls — BRCA1 and BRCA2 in Homo sapiens (default species, may be omitted) and in Pan troglodytes (non-default → required).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 712 OUT445
✓	ProteinsInThreeCompartmentsBFCL/parallel_109: three cellbio_get_proteins calls — nucleus/mitochondria/cytoplasm, each with include_description=true (prompt asks for "a brief description of each protein").1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 567 OUT600
Tool Calling - Parallel Calls: Physical & Life Sciences (set 3) 13 passed	100.0%	IN113 625 OUT5 396
✓	AnalyzeDnaForThreeMutationTypesBFCL/parallel_114: six analyze_dna_sequence calls — two sequences ("AGCTTAGCTA" and "AGCTTAGGCTA") against reference "AGCTTAGCTA", each checked for insertion (default, may be omitted), deletion, and substitution mutations.1 AI query 6 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL	Passed	IN9 205 OUT749
✓	DiabetesPredictionForFourPeopleBFCL/parallel_113: four diabetes_prediction calls for four people with distinct weight/height/activity.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 966 OUT494
✓	EntropyChangeIsothermalAndNotBFCL/parallel_104: two calculate_entropy_change calls — same 300->350 K and heat_capacity 4.18, once isothermal (the default true, may be omitted) and once not isothermal (false, required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 628 OUT392
✓	ForestGrowthForTwoRegionsBFCL/parallel_120: two forest_growth_forecast calls — Amazon Rainforest over 10 years and Boreal Forests of Canada over 20 years, excluding human impact (false = the default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 519 OUT213
✓	GenerateFourDnaSequencesBFCL/parallel_123: four generate_DNA_sequence calls — length 500 with a single-nucleotide preference of A, T, C, G respectively.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 869 OUT432
✓	GeneticSimilarityInTwoFormatsBFCL/parallel_115: four genetics_calculate_similarity calls — human vs chimpanzee and human vs gorilla, each in percentage (default, may be omitted) and fraction. species1 accepts "human"/"Human".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 814 OUT400
✓	GenotypeFrequenciesForThreeGenotypesBFCL/parallel_116: three calculate_genotype_frequency calls — allele frequency 0.7, genotypes AA/Aa/aa.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 683 OUT381
✓	IdentifyFourBirdsBFCL/parallel_119: four identify_bird calls — small blue/forest (default size, may be omitted), large black/lake, medium brown/desert, large green/tropical rainforest.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 922 OUT518
✓	MutationTypeForTwoSnpsBFCL/parallel_112: two mutation_type_find calls — rs123456 in Homo sapiens (default species, may be omitted) and rs7891011 in Canis lupus familiaris (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 541 OUT294
✓	PopulationDensityForChinaTwoYearsBFCL/parallel_117: two calculate_density calls (country/population schema) — China in 2000 (pop 1.267 billion) and 2010 (pop 1.341 billion), land area 9.597 million km² both times.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 604 OUT309
✓	PrecipitationStatsForThreeTimeFramesBFCL/parallel_118: three ecology_data_precipitation_stats calls — Amazon rainforest for six_months, year, five_years.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 614 OUT319
✓	TurtlePopulationForTwoYearsBFCL/parallel_121: two ecology_get_turtle_population calls — Galapagos Islands in 2015 and 2020, both with species info requested (species=true; years stated explicitly → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 534 OUT483
✓	VehicleEmissionsForThreeVehiclesBFCL/parallel_122: three calculate_vehicle_emission calls — 15000 miles each: gas with default factor 1.4 (may be omitted), diesel with factor 2.7, EV with factor 0.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 726 OUT412
Tool Calling - Parallel Calls: Services & Lookups 12 passed	100.0%	IN98 440 OUT3 780
✓	CalculateShortestAndScenicDistancesForTwoLegsBFCL/parallel_190: four calculate_shortest_distance calls — {NYC->LA, LA->Miami} x {Shortest, Scenic}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 372 OUT532
✓	FetchNewsForTwoTopicsInEuropeBFCL/parallel_192: two get_news calls — Climate Change and Artificial Intelligence, 5 articles each, region Europe. Region "Europe" explicitly requested -> required (accept "Europe"/"EU").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 053 OUT211
✓	FindLibrariesWithFacilitiesInThreeCitiesBFCL/parallel_191: three public_library_find_nearby calls — NY[Reading Room,Fiction], LA[Wi-Fi,Children Section], Chicago[Cafe,Reading Room]. Facilities compared as an unordered pair.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 247 OUT333
✓	FindParksWithAmenitiesInThreeCitiesBFCL/parallel_189: three parks_find_nearby calls — NY[Tennis Court,Picnic Area], LA[Playground,Running Track], Chicago[Tennis Court,Playground]. Amenities compared as an unordered pair.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 252 OUT351
✓	FindUpcomingEventsInThreeCitiesBFCL/parallel_194: three event_finder_find_upcoming calls — jazz/LA(14d), rock/Chicago(10d), classical/Boston(7d). All day windows explicitly stated -> days_ahead required (7 for Boston, which also equals the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 265 OUT334
✓	GetCurrentWeatherForFourCitiesBFCL/parallel_199: four get_current_weather calls — New York, Los Angeles, London, Tokyo, with both temperature and humidity included. Prompt explicitly asks to include both -> flags == true (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 347 OUT324
✓	GetDetailedBriefsForTwoMoviesBFCL/parallel_195: two movie_details_brief calls — Inception and The Dark Knight, extra_info true. Prompt asks for Director/Cast/Awards -> extra_info == true required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 031 OUT415
✓	GetLawsuitDetailsForTwoCasesBFCL/parallel_196: two get_lawsuit_details calls — case 12345 (NY Supreme Court, with verdict) and case 67890 (LA Superior Court, no verdict). "would like the verdict" -> true; "do not need" -> null/false.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 180 OUT245
✓	GetLawsuitInfoForTwoCasesByYearBFCL/parallel_197: two lawsuit_info calls — 12345ABC (2018, New York) and 67890XYZ (2019, California). Years are explicitly stated -> required. First location "New York" may be omitted (matches the default).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 171 OUT257
✓	GetStockPricesForTwoCompaniesOverDaysBFCL/parallel_179: two get_stock_price (by-days variant) calls — Apple on NASDAQ for 10 days, Microsoft on NYSE for 15 days. Both exchanges are explicitly named -> required (NASDAQ / NYSE). The tool's 'company' field is a ticker symbol; company names are accepted as synonyms.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 054 OUT227
✓	SearchLawsuitsForTwoEntitiesInCaliforniaBFCL/parallel_198: two lawsuit_search calls — Google/Santa Clara and Facebook/San Mateo, state California. Prompt states "both in the state of California" -> state required (accept "California"/"CA").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 091 OUT240
✓	SendTwoEmailsWithCcAndBccBFCL/parallel_193: two send_email calls — Project Update to john.doe (cc manager, bcc hr) and Meeting Reminder to jane.doe (cc assistant, bcc hr). Bodies matched exactly.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 377 OUT311
Tool Calling - Parallel Calls: Statistics & Finance 13 passed	100.0%	IN117 665 OUT5 117
✓	CalculateBinomialProbabilitiesForBasketballShotsBFCL/parallel_163: three calc_binomial_prob calls — (10,6,0.6), (10,6,0.5), (15,6,0.5).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 161 OUT357
✓	CalculateCompoundInterestForThreePeriodsBFCL/parallel_170: three calculate_compound_interest calls — principal 5000, rate 0.025, time 2/3/5. Prompt states "compounded quarterly, 4 times in a year" -> n == 4 required (tightened; also ground truth).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 114 OUT428
✓	CalculateDiscountedCashFlowForTwoBondsBFCL/parallel_169: two calculate_discounted_cash_flow calls — (50,5,0.05) and (60,7,0.04). Prompt explicitly states "face value of the bond is $1000" for both -> face_value == 1000 required (ground truth also accepts omission via the default, but an explicitly-stated value must be supplied).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 998 OUT363
✓	CalculateEventProbabilitiesForCardDeckBFCL/parallel_162: three probability_of_event calls — Ace 4/52 as decimal, heart 13/52 as decimal, red card 26/52 as ratio. format_as_ratio: "as a decimal" -> false or omitted; "as a ratio" -> true (required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 146 OUT338
✓	CalculateQuarterlyDividendForThreeScenariosBFCL/parallel_168: three finance_calculate_quarterly_dividend_per_share calls — (5M,2M), (6M,2M), (6M,2.5M).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 184 OUT406
✓	CalculateReturnOnEquityForTwoCompaniesBFCL/parallel_171: two calculate_return_on_equity calls — (1M,5M,200k) and (2M,10M, no dividends). Second company "did not pay any dividends" -> dividends_paid null or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 959 OUT511
✓	CalculateRoiForThreeStocksBFCL/parallel_174: three calculate_return_on_investment calls — (150,180,20), (200,210,30), (250,300,40). Dividends are explicitly stated per stock -> required.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 100 OUT380
✓	ComputeRSquaredForTwoRegressionModelsBFCL/parallel_167: two linear_regression_get_r_squared calls on the same CSV — swap education/credit_score as dependent variable. Independent-variable order follows the prompt/ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 934 OUT294
✓	PredictFutureValueForTwoInvestmentsBFCL/parallel_172: two finance_predict_future_value calls — (5000,0.05,10) and (7000,0.04,15). Prompt states "compound interest annually" -> compounding_periods_per_year == 1 required (tightened; also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 944 OUT336
✓	PredictHousePricesForThreeHousesBFCL/parallel_166: three predict_house_price calls — NY 2500/3/2000, LA 3000/3/2005, Chicago 2000/2/1995. The second house is described as "similar" with only area/year/location changed, so rooms == 3 per ground truth.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 151 OUT547
✓	PredictProfitForTwoFundsBFCL/parallel_173: two investment_predictProfit calls — (5000,0.07,5) and (8000,0.05,7).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 920 OUT309
✓	RunChiSquaredTestForTwoContingencyTablesBFCL/parallel_164: two chi_squared_test calls — tables [45,55,35,65] and [30,70,50,50], alpha 0.05. Prompt states "significance level of 0.05 for both" -> alpha == 0.05 required (tightened). Table order is fixed.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 904 OUT335
✓	RunTTestAtTwoSignificanceLevelsBFCL/parallel_165: two t_test calls — same datasets, alpha 0.05 then 0.01 (both stated -> required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 150 OUT513
Tool Calling - Simple: Arts, Venues & Music (set 1) 8 passed	100.0%	IN50 817 OUT1 670
✓	SimpleConcertDetailsBeyonceBFCL/simple_python_286 (concert.get_details): artist=Beyonce, location=San Diego, date=April 2022.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 362 OUT92
✓	SimpleConcertInfoPopNyBFCL/simple_python_284 (concert_info.get): location=New York, date=next month, genre=Pop.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 337 OUT94
✓	SimpleGenerateMelodyC4BFCL/simple_python_291 (music_generator.generate_melody): key=C, start_note=C4, length=16, tempo=120 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 385 OUT175
✓	SimpleKeySignatureCsharpBFCL/simple_python_298 (music_theory.key_signature): key='C#', scale_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 322 OUT145
✓	SimpleMuseumHoursLouvreBFCL/simple_python_273 (museum.get_hours): name='Louvre Museum', location=Paris, day omitted or Monday.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 329 OUT137
✓	SimplePaintNeededWall30x12BFCL/simple_python_259 (calculate_paint_needed): coverage_rate=400, length=30, height=12.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL	Passed	IN6 385 OUT138
✓	SimpleReligionHistoryChristianity16cBFCL/simple_python_252 (get_religion_history): religion=Christianity, century=16, sort_by=importance, count=10.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 362 OUT602
✓	SimpleSculptureInfoPlensaBFCL/simple_python_263 (get_sculpture_info): artist_name='James Plensa', detail=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 335 OUT287
Tool Calling - Simple: Arts, Venues & Music (set 2) 8 passed	100.0%	IN50 567 OUT1 495
✓	SimpleComposeMelodyCFGBFCL/simple_python_292 (compose_melody): progression=[C,F,G], measures=4, instrument=Piano or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 323 OUT111
✓	SimpleConcertFindDetailsWeekndBFCL/simple_python_290 (concert.find_details): artist=The Weeknd, month=December, year=2022 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 301 OUT110
✓	SimpleFindConcertRockChicagoBFCL/simple_python_285 (find_concert): location=Chicago, price=100, genre=Rock.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 311 OUT109
✓	SimpleMuseumInfoMetBFCL/simple_python_274 (museum_info): museum_name=Metropolitan Museum of Art, info_type=opening_hours or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 299 OUT149
✓	SimpleMusicalScaleCsharpBFCL/simple_python_299 (musical_scale): key='C#'/'C sharp', scale_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 290 OUT96
✓	SimplePaintRequirementWallExclusionBFCL/simple_python_260 (paint_requirement.calculate): area {width=20,height=12}, paint_coverage=350, exclusion {type=window, area=15}.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN6 409 OUT205
✓	SimpleReligionHistoryChristianity300To400BFCL/simple_python_254 (get_religion_history): religion=Christianity, start_year=300, end_year=400, event_type=all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 326 OUT532
✓	SimpleSculptureDetailsDavidBFCL/simple_python_264 (sculpture.get_details): artist=Michelangelo, title=David, detail=size.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 308 OUT183
Tool Calling - Simple: Arts, Venues & Music (set 3) 9 passed	100.0%	IN60 251 OUT1 597
✓	SimpleChordProgressionKeyBFCL/simple_python_297 (music.theory.chordProgression): progression=[I,V,vi,IV], bools any.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 704 OUT147
✓	SimpleCompoundInterest10kBFCL/simple_python_269 (calculate_compound_interest): principle=10000, interest_rate=0.05, time=10, compounds_per_year=1 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 740 OUT297
✓	SimpleConcertSearchClassicalLaBFCL/simple_python_287 (concert.search): genre=classical, location=Los Angeles/LA, date='this weekend', price_range=cheap.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 696 OUT108
✓	SimpleCreateMixCMajorBFCL/simple_python_293 (music_composer.create_mix): scale='C Major', note_duration=quarter, track_length=180.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 719 OUT97
✓	SimpleGenerateCircleImageRedBFCL/simple_python_256 (generate_circle_image): radius=50, color=Red, background=white or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 688 OUT91
✓	SimpleMuseumWorkingHoursLouvreBFCL/simple_python_276 (museum_working_hours.get): museum=Louvre, location=Paris, day=Monday or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 674 OUT95
✓	SimpleRetrieveReligionInfoBuddhismBFCL/simple_python_253 (retrieve_religion_info): religion_name=Buddhism, detail_level=full.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 669 OUT565
✓	SimpleSculptureSearchChicago19cBFCL/simple_python_265 (sculpture_search): location=Chicago, time_frame='19th century', material=all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 686 OUT85
✓	SimpleTimeDifferenceSfSydneyBFCL/simple_python_250 (get_time_difference): place1=San Francisco/SF, place2=Sydney.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL	Passed	IN6 675 OUT112

Tool Calling - Irrelevance: Architecture & Artwork 9 passed

100.0%

IN31 151 OUT2 110

✓

NoToolForApartmentRentCostBFCL/irrelevance_158: apartment rental cost asked; only a new-construction cost estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 463 OUT98

✓

NoToolForBestJazzSingerBFCL/irrelevance_164: best Jazz singer asked; only a top-instrument-players search (singers are not instrument players) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 458 OUT474

✓

NoToolForCathedralCeilingDesignBFCL/irrelevance_157: how to design a cathedral ceiling asked; only a lookup of facts about an existing building is available (retrieves data, does not advise on design) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 459 OUT418

✓

NoToolForHolyBookDefinitionBFCL/irrelevance_144: definition of a holy book asked; only a search-within-a-named-holy-book tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 457 OUT82

✓

NoToolForLectureTeacherBFCL/irrelevance_167: teacher of upcoming lectures asked; only an upcoming-concert details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 460 OUT47

✓

NoToolForMostFamousSculptureBFCL/irrelevance_153: most famous sculpture asked; only a sculpting-tools lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 465 OUT253

✓

NoToolForPianoMaintenanceTipsBFCL/irrelevance_166: piano maintenance tips asked; only an instrument rental-prices lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 460 OUT437

✓

NoToolForScreamArtistBFCL/irrelevance_159: artist behind 'The Scream' asked. NOTE: the only tool (artwork_search) requires a museum_location that the user did not provide, so it cannot be correctly called; BFCL labels this irrelevant. Kept faithfully with ground truth = no tool call (flagged as borderline).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 464 OUT18

✓

NoToolForSculptureRockTypeBFCL/irrelevance_152: which rock suits a garden sculpture asked; only a sculpture 3D-model creator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 465 OUT283

Tool Calling - Irrelevance: Buildings, Sculpture & Music 7 passed1 skipped

100.0%

IN23 003 OUT1 220

–

NoToolForCelloInstrumentTypeBFCL/irrelevance_165: instrument classification of a cello asked. NOTE: BFCL labels this irrelevant, but the only tool (get_instrument_info with detail=type) arguably answers it; flagged for review as a borderline BFCL item. Kept faithfully with ground truth = no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_instrument_info' with detail='type' answers 'what type of instrument is a cello' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForDeclarationSignersBFCL/irrelevance_137: who signed the declaration asked; only a historical-event DATE lookup is available (returns a date, not the signatories) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 284 OUT74

✓

NoToolForEiffelTowerHeightBFCL/irrelevance_156: Eiffel Tower height asked; only an architecture-plan generator (creates a new design, does not retrieve facts) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 288 OUT32

✓

NoToolForFamousDancersBFCL/irrelevance_162: famous dancers asked; only a museum-visual-artists lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 290 OUT457

✓

NoToolForReformationInitiatorBFCL/irrelevance_145: who initiated the Reformation asked; only a religious-event YEAR lookup is available (returns a year, not a person) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 283 OUT80

✓

NoToolForScientistDefinitionBFCL/irrelevance_143: definition of "scientist" asked; only a named-historical-figure info lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 280 OUT167

✓

NoToolForSellingGuitarBFCL/irrelevance_163: how to sell a guitar asked; only an instrument-tuning tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 285 OUT247

✓

NoToolForSimilarBlueColorBFCL/irrelevance_148: a color similar to blue asked; only a complementary-color tool (opposite, not similar) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 293 OUT163

Tool Calling - Irrelevance: Cell Biology & Genetics 8 passed2 skipped

100.0%

IN30 331 OUT2 070

✓

NoToolForClottingBloodCellsBFCL/irrelevance_32: blood cell type responsible for clotting asked; only a nucleus-count-based cell-type classifier is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 792 OUT50

✓

NoToolForCommonAncestorBFCL/irrelevance_41: common ancestor of two animals asked (phylogenetics knowledge); only a fibonacci-sequence calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 791 OUT209

✓

NoToolForEvolutionaryAlgorithmInGameAIBFCL/irrelevance_43: how to apply an Evolutionary Algorithm in game AI asked (advice); only a creature-evolution simulator is available (wrong context).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 792 OUT525

✓

NoToolForGeneVariantFrequencyBFCL/irrelevance_36: gene-variant frequency in a population asked; only a dominant-trait-from-alleles tool is available (the true-match variant-frequency tool is deliberately isolated in A4).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 797 OUT194

–

NoToolForIdentifyingGeneticSequenceBFCL/irrelevance_33: interpret a genetic code sequence asked; only a species-identification-from-sequence tool is available (returns a species, not what the sequence means).0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'identify_species' (identifies a species from a genetic sequence) plausibly answers the query about a DNA sequence, so a competent model may call it - BFCL-mislabeled/borderline-relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForIncreasingDeerPopulationBFCL/irrelevance_39: how-to advice on growing an animal population asked; only a population-growth CALCULATOR (needs numeric rates) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 791 OUT246

✓

NoToolForLionMatingProcessBFCL/irrelevance_35: mating process of a species asked; only a dominant/recessive genetic-traits lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 788 OUT255

✓

NoToolForMitochondriaCountBFCL/irrelevance_29: number/count of mitochondria asked; only a cell-part FUNCTION lookup is available (wrong query direction).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 792 OUT220

✓

NoToolForPandaEvolutionaryHistoryBFCL/irrelevance_42: evolutionary history asked (a narrative fact); only a biodiversity-index calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 788 OUT371

–

NoToolForWhaleGeneSequenceBFCL/irrelevance_44: the actual gene sequence of a species asked; only a gene-sequence GENERATOR (needs numeric mutation rate + duration) is available.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'gene_sequencer' (generate gene sequences for evolutionary changes) directly answers the query - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Irrelevance: Color, Geometry & Population 11 passed

100.0%

IN40 858 OUT1 271

✓

NoToolForColorInComputerVisionBFCL/irrelevance_131: color meaning in computer vision asked; only a color-in-personality-psychology analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 714 OUT253

✓

NoToolForDeclarationSigningDateBFCL/irrelevance_135: signing date asked; only date-arithmetic (add days) and age tools are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 712 OUT86

✓

NoToolForElectricityDiscovererBFCL/irrelevance_140: who discovered electricity asked; only an electric-current calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 708 OUT260

✓

NoToolForHexColorMixBFCL/irrelevance_146: mixing two hex colors asked; only a prophet-details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 724 OUT191

✓

NoToolForIndependenceSigningDateBFCL/irrelevance_138: signing date asked; only an age calculator (needs birthdate + current date) and a date-adder are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 712 OUT68

✓

NoToolForMostVisitedMarketBFCL/irrelevance_161: most visited market asked; only a museum visitation-statistics lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 714 OUT79

✓

NoToolForNbaFinalWinnerBFCL/irrelevance_133: NBA final winner asked; only a historical-battle details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 715 OUT36

✓

NoToolForPicassoPaintingYearBFCL/irrelevance_151: year of a Picasso painting asked; only a sculpture-dimensions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 713 OUT78

✓

NoToolForRelativityAuthorBFCL/irrelevance_142: who proposed special relativity asked; only a chemical-element properties lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 717 OUT42

✓

NoToolForSculptureCreatorBFCL/irrelevance_155: sculpture creator asked; only a max-seating-capacity calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 713 OUT13

✓

NoToolForWhichColorsToMixBFCL/irrelevance_150: which colors to mix for a shade asked; only an RGB-to-Pantone converter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 716 OUT165

Tool Calling - Irrelevance: Cooking, Currency & Markets 7 passed1 failed

87.5%

IN29 247 OUT2 131

×

NoToolForBakerySourdoughBFCL/irrelevance_223: a BAKERY selling sourdough asked; only a grocery-shop product finder available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 535 OUT299

✓

NoToolForChickenRecipeBFCL/irrelevance_211: the RECIPE to cook chicken asked; only a cook-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 243 OUT273

✓

NoToolForChocolateCakeIngredientsBFCL/irrelevance_220: cake INGREDIENTS asked; only a cooking-TEMPERATURE tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 246 OUT124

✓

NoToolForCurrencyToGbpBFCL/irrelevance_233: CURRENCY conversion asked; only a WEATHER tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 248 OUT88

✓

NoToolForHowToBoilAnEggBFCL/irrelevance_212: best WAY to boil an egg asked; only a boiling-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 245 OUT201

✓

NoToolForLasagnaRecipeBFCL/irrelevance_219: the RECIPE for lasagna asked; only a recipe-CALORIE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 243 OUT539

✓

NoToolForLegExercisesBFCL/irrelevance_221: leg EXERCISES asked; only a grocery-list generator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 243 OUT449

✓

NoToolForPetStoreBFCL/irrelevance_224: a PET store asked; only a GROCERY-store locator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 244 OUT158

Tool Calling - Irrelevance: Dates & Elements 9 passed1 failed

90.0%

IN40 022 OUT1 554

✓

NoToolForHydrogenPropertiesBFCL/irrelevance_141: chemical properties of Hydrogen asked; only a scientist-contributions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 628 OUT319

✓

NoToolForLargestPlanetBFCL/irrelevance_139: largest planet asked; only a star-info lookup (needs a star name) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 628 OUT143

✓

NoToolForMostImportantProphetBFCL/irrelevance_147: most important prophet asked; only a two-color mixer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 628 OUT101

×

NoToolForMuseumVisitFrequencyBFCL/irrelevance_160: museum member visit frequency asked; only a single most-frequent-visitor lookup is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN7 365 OUT292

✓

NoToolForPantoneOfSkyBlueBFCL/irrelevance_149: Pantone code of sky blue asked; only a paint-proportions calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 630 OUT143

✓

NoToolForPersonalityTypeBFCL/irrelevance_130: user asks what personality type they are; only a big-five calculator (which needs the user's rated answers) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 626 OUT87

✓

NoToolForStadiumSeatingCapacityBFCL/irrelevance_154: stadium seating capacity asked; only a sculpture-creator lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 629 OUT97

✓

NoToolForVicePresidentBFCL/irrelevance_136: Vice President asked; only a US-president-by-year lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 628 OUT46

✓

NoToolForWaterlooCasualtiesBFCL/irrelevance_132: casualty count of a battle asked; only a historical-event date lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 630 OUT240

✓

NoToolForWorldCupWinnerBFCL/irrelevance_134: sports World Cup winner asked; only a historical-battle outcome predictor is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 630 OUT86

Tool Calling - Irrelevance: Electromagnetism, Thermo & Data 10 passed

100.0%

IN34 065 OUT1 355

✓

NoToolForElectronRestMassBFCL/irrelevance_23: mass of an electron asked (a knowledge fact); only magnetic-field / current tools are available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 399 OUT46

✓

NoToolForFlightDurationBFCL/irrelevance_20: flight duration asked; only a currency converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 402 OUT146

✓

NoToolForFreezingPointBFCL/irrelevance_25: freezing point asked; only boiling-point calculators are available (wrong phase transition).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 409 OUT152

✓

NoToolForGasAmountGeneratedBFCL/irrelevance_26: amount of gas generated asked; only a gas-PRESSURE calculator is available (wrong quantity).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 422 OUT96

✓

NoToolForGiniCoefficientBFCL/irrelevance_48: Gini coefficient asked; only a GDP-fetch tool is available (different economic indicator).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 402 OUT231

✓

NoToolForHeatingWaterBFCL/irrelevance_27: heat energy to warm water asked. NOTE: the original tool (calculate_heat) is a fair match for this query (only the well-known specific heat is missing), which would make "no call" ambiguous; per human-baseline fairness it is deliberately NOT shown here (kept as a distractor in A3), so no visible tool fits and "no call" is unambiguously correct.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 413 OUT315

✓

NoToolForHexagonSidesBFCL/irrelevance_28: number of sides of a hexagon asked (basic geometry knowledge); only a boiling-point tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 400 OUT9

✓

NoToolForSymbolicMagneticFieldBFCL/irrelevance_22: magnetic field asked with SYMBOLIC values (distance 'r', current 'I'); the two magnetic-field tools require concrete numeric distance + current, so there is nothing to pass -> no fit.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 413 OUT150

✓

NoToolForTimeDifferenceBFCL/irrelevance_49: time-zone difference asked; only a geo-coordinate lookup tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 402 OUT184

✓

NoToolForWorldSeriesWinnerBFCL/irrelevance_53: sports trivia asked; only a SQL database-query runner is available (no database/query provided).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 403 OUT26

Tool Calling - Irrelevance: Entertainment, Sports & Knowledge 9 passed

100.0%

IN31 428 OUT1 423

✓

NoToolForBasketballResultBFCL/irrelevance_84: basketball game result asked; only a stock-price retrieval tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 492 OUT58

✓

NoToolForBestSoccerPlayerBFCL/irrelevance_79: best current soccer player asked; only a stock-market prediction tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 490 OUT356

✓

NoToolForCaliforniaGovernorBFCL/irrelevance_87: current Governor of California asked; only a court-case filing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 488 OUT41

✓

NoToolForCrimeThrillerMovieListBFCL/irrelevance_88: best crime-thriller movies of 2020 asked; only a document forgery detector is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 497 OUT203

✓

NoToolForFifaWorldCupWinnerBFCL/irrelevance_80: FIFA World Cup 2010 winner asked; only a stock-ticker lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 493 OUT48

✓

NoToolForFrenchTranslationBFCL/irrelevance_61: French translation of a word asked; only an SVM-model tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 492 OUT98

✓

NoToolForHorrorMovieListBFCL/irrelevance_77: horror movie recommendations asked; only an investment future-value tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 491 OUT413

✓

NoToolForPresidentialElectionWinnerBFCL/irrelevance_85: 2020 presidential election winner asked; only a criminal-case detail lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 493 OUT40

✓

NoToolForSportsNewsBFCL/irrelevance_76: latest sports news asked; only a portfolio investment-trend analyzer is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 492 OUT166

Tool Calling - Irrelevance: Finance & Economics 8 passed

100.0%

IN26 842 OUT1 313

✓

NoToolForAverageHousePriceBFCL/irrelevance_69: market-wide average house price asked; only a regression predictor (needs a trained model + feature vector, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 348 OUT124

✓

NoToolForCompoundInterestBFCL/irrelevance_70: compound-interest computation asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 374 OUT492

✓

NoToolForGoldPriceBFCL/irrelevance_78: today's gold spot price asked; only a bond-price calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 350 OUT84

✓

NoToolForHorrorMovieCatalogBFCL/irrelevance_81: horror movie recommendations asked; only a historical stock-price fetcher is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 351 OUT336

✓

NoToolForLossProjectionBFCL/irrelevance_73: a loss projection asked; only a revenue-prediction tool is available (revenue != loss).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 353 OUT102

✓

NoToolForPoundToKilogramBFCL/irrelevance_75: a mass unit conversion asked; only a portfolio-returns calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 349 OUT14

✓

NoToolForProfitMarginBFCL/irrelevance_71: profit margin asked; only an ROI calculator is available (ROI != margin) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 371 OUT109

✓

NoToolForStockNewsBFCL/irrelevance_82: stock news asked; only a capital-gains calculator is available (gains != news) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 346 OUT52

Tool Calling - Irrelevance: Finance Near-Matches 8 passed1 failed

88.9%

IN34 473 OUT1 843

✓

NoToolForBitcoinPriceBFCL/irrelevance_56: current bitcoin price asked; only a generic database record-select tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 428 OUT66

✓

NoToolForBookstoreLookupBFCL/irrelevance_55: which online bookstore sells a title asked; only an add-to-cart tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 433 OUT89

×

NoToolForBusinessRateOfReturnBFCL/irrelevance_74: rate of return asked; only a net-profit calculator is available (profit != rate of return).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 983 OUT321

✓

NoToolForCoinFlipProbabilityBFCL/irrelevance_66: probability of a coin landing heads asked; only a standard-deviation tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 433 OUT63

✓

NoToolForCurrentInterestRateBFCL/irrelevance_83: the current market interest rate asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 427 OUT122

✓

NoToolForExternalRateOfReturnBFCL/irrelevance_72: EXTERNAL rate of return asked; only an internal-rate-of-return (IRR) tool is available (ERR != IRR, even though cash flows are given) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 457 OUT234

✓

NoToolForFinalPriceWithoutBasePriceBFCL/irrelevance_60: final price after discount+tax asked, but the original price is never given, so the (domain-matching) calculateFinalPrice cannot be correctly called -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 447 OUT52

✓

NoToolForFrontendDevelopmentBFCL/irrelevance_62: how to build an e-commerce frontend asked; only a recommender-model builder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 433 OUT861

✓

NoToolForHighestGrossingMovieBFCL/irrelevance_54: highest-grossing movie of all time asked; only a movie search (requires a specific title + year, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 432 OUT35

Tool Calling - Irrelevance: Food, Time & Distance 4 passed1 skipped

100.0%

IN10 840 OUT364

✓

NoToolForCakeBakingMachineBFCL/irrelevance_218: a baking MACHINE/appliance asked; only a recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 708 OUT37

✓

NoToolForDrivingTravelTimeBFCL/irrelevance_215: driving TRAVEL TIME asked; only a restaurant-bill tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 712 OUT148

✓

NoToolForLondonTimezoneNameBFCL/irrelevance_226: the NAME of London's timezone asked; only a local-TIME-by-timezone tool available (it needs the timezone as input and returns a time, not the timezone name) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 704 OUT54

–

NoToolForPizzaInBostonBFCL/irrelevance_213: a place for pizza asked; a cuisine+location restaurant_finder is available. REVIEW FLAG (fairness): restaurant_finder(cuisine=pizza/Italian, location=Boston) plausibly answers this, so a competent human might legitimately call it. BFCL labels it irrelevant; kept as such and marked Hard because the intended "no call" is debatable. No OTHER query here is a restaurant search, so the tool cannot wrongly satisfy the file's other cases.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'restaurant_finder' (cuisine+location) answers 'good place for pizza in Boston' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForTriangleHypotenuseBFCL/irrelevance_238: the HYPOTENUSE of a triangle asked; only a geo lat/long distance calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 716 OUT125

Tool Calling - Irrelevance: Geo, Travel & Sentiment 8 passed1 skipped

100.0%

IN27 187 OUT930

✓

NoToolForAverageTemperatureBmiBFCL/irrelevance_110: an average December temperature asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 398 OUT51

✓

NoToolForCaseLawyerBFCL/irrelevance_95: the lawyer of a case asked; only a case-outcome retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 401 OUT106

✓

NoToolForCurrentTimeSunriseBFCL/irrelevance_102: the current time asked; only a sunrise/sunset time calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 397 OUT102

✓

NoToolForFamousQuoteBFCL/irrelevance_119: a famous quote asked; only a sentiment-analysis tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 395 OUT30

✓

NoToolForFastestRouteBFCL/irrelevance_114: the fastest route asked; only a closest-airport finder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 399 OUT126

–

NoToolForRestaurantsInLondonBFCL/irrelevance_113: restaurants in London asked; only a generic points-of-interest locator is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'find_pois' (locate points of interest by category) answers 'restaurants in London' (restaurants are POIs) - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForSportsBrainImpactBFCL/irrelevance_120: the neurological impact of sports asked; only a caffeine neurological-impact tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 398 OUT246

✓

NoToolForTrafficConditionBFCL/irrelevance_112: the current traffic condition asked; only an address geocoder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 397 OUT114

✓

NoToolForTravelTimeByCarBFCL/irrelevance_115: travel time by car asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 402 OUT155

Tool Calling - Irrelevance: Legal & Case Lookup 10 passed

100.0%

IN35 527 OUT1 197

✓

NoToolForAppleCaloriesBFCL/irrelevance_100: calories in an apple asked; only a litigation-cost calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 550 OUT94

✓

NoToolForBrainWeightBFCL/irrelevance_122: average human brain weight asked; only a per-patient neural-activity tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 552 OUT101

✓

NoToolForCaseJudgementBFCL/irrelevance_93: a case judgement asked; only a law-firm impactful-cases lister is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 550 OUT59

✓

NoToolForChineseRestaurantBFCL/irrelevance_97: a Chinese-restaurant recommendation asked; only a lawsuit-filing tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 553 OUT182

✓

NoToolForMovieRecommendationBFCL/irrelevance_116: a movie recommendation asked; only a word-count tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 551 OUT102

✓

NoToolForNbaChampionBFCL/irrelevance_118: the 1996 NBA champion asked; only an NBA playoff-brackets display tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 553 OUT48

✓

NoToolForPaintingEiffelSettlementBFCL/irrelevance_98: how long to paint the Eiffel Tower asked; only a lawsuit-settlement estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 554 OUT147

✓

NoToolForSecurityGuardSearchBFCL/irrelevance_92: finding a security guard asked; only a lawyer-search tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 560 OUT53

✓

NoToolForSphereVolumeBFCL/irrelevance_104: sphere volume asked; only a circular-park area tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 554 OUT176

✓

NoToolForTrafficLawsBFCL/irrelevance_99: traffic laws asked; only a lawsuit-search tool is available (lawsuits, not laws) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 550 OUT235

Tool Calling - Irrelevance: Math & Science Distractors 8 passed

100.0%

IN25 252 OUT1 089

✓

NoToolForDefiniteIntegralBFCL/irrelevance_9: definite integral asked; only an income-tax calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 159 OUT198

✓

NoToolForIntegralOfCubeBFCL/irrelevance_8: definite integral asked; only a string-to-int converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 152 OUT198

✓

NoToolForLatestMoviesBFCL/irrelevance_15: latest movie releases asked; only a physics final-velocity tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 146 OUT106

✓

NoToolForPerpendicularSlopeBFCL/irrelevance_3: slope of a perpendicular line asked; only a critical-points finder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 164 OUT147

✓

NoToolForPrimeFactorsBFCL/irrelevance_13: prime factorization asked; only a compound-interest calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 149 OUT97

✓

NoToolForRectangleAreaBFCL/irrelevance_6: rectangle area asked; only a circle-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 160 OUT74

✓

NoToolForTriangleAreaBFCL/irrelevance_0: triangle area asked; only a BMI tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 160 OUT82

✓

NoToolForUpwardAccelerationBFCL/irrelevance_14: acceleration asked (it is just gravity); the tempting max-height and final-velocity tools compute neither the acceleration nor what is asked -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 162 OUT187

Tool Calling - Irrelevance: Math & Statistics 6 passed

100.0%

IN18 082 OUT1 298

✓

NoToolForBinomialMeanBFCL/irrelevance_67: the MEAN of a binomial experiment asked; only p-value tools are available (they return a p-value, not the mean) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 028 OUT68

✓

NoToolForCivilLawComponentsBFCL/irrelevance_90: the components/categories of Civil Law asked; only a noise-complaint filing tool is available (filing a complaint does not explain law) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 000 OUT214

✓

NoToolForCoinTossMaxHeadsBFCL/irrelevance_63: how many heads CAN be obtained from tossing 3 coins (a count/maximum, not a probability) asked; only a probability-of-an-event calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 004 OUT150

✓

NoToolForNullHypothesisStdDevBFCL/irrelevance_68: the STANDARD DEVIATION of a null-hypothesis test asked; only p-value tools are available (they return a p-value, not a standard deviation) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 038 OUT549

✓

NoToolForPowerOfTwoBFCL/irrelevance_59: raising a number to a power asked; only a linear-equation solver is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 003 OUT20

✓

NoToolForQuadraticEquationBFCL/irrelevance_57: solving a quadratic equation asked; only a genetic-algorithm optimizer and a LINEAR equation solver are available (a quadratic is not linear) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 009 OUT297

Tool Calling - Irrelevance: Medical, Social & Personality 6 passed3 skipped

100.0%

IN20 464 OUT1 445

✓

NoToolForBigMacCaloriesBFCL/irrelevance_123: calories of a Big Mac asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 409 OUT111

✓

NoToolForEconomicHappinessEffectBFCL/irrelevance_126: the effect of economic status on happiness asked; only a happiness-index lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 411 OUT296

✓

NoToolForFriendMbtiBFCL/irrelevance_129: a friend's MBTI asked; only a personality-type evaluator (needs trait percentages) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 409 OUT140

–

NoToolForLatestTechTrendBFCL/irrelevance_124: the latest technology trend asked; only a trending-topics-by-category tool is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_social_trends' (trending topics by category) answers 'latest trend in technology' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForMajorAirportsBFCL/irrelevance_117: major US airports asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 412 OUT328

–

NoToolForMotorNeuronDiseaseBFCL/irrelevance_121: motor neuron disease info asked; only a generic disease-info retriever is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'medical_records_get_disease_info' directly answers 'information on motor neuron diseases' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForPopularBooksBFCL/irrelevance_125: popular books by an author asked; only a recent-tweets retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 411 OUT438

–

NoToolForTwitterMoodBFCL/irrelevance_127: overall twitter mood asked; only a twitter sentiment-analysis tool is available -> BFCL ground truth is no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'sentiment_analysis_twitter' directly answers 'general mood of twitter regarding the new iPhone' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForVegetableServingsBFCL/irrelevance_128: daily vegetable servings asked; only a personality-test scorer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 412 OUT132

Tool Calling - Irrelevance: Mixed Distractors 8 passed

100.0%

IN26 008 OUT1 249

✓

NoToolForAirQualityBFCL/irrelevance_40: current air quality asked; only a plant-biomass calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 252 OUT118

✓

NoToolForFavoriteRestaurantBFCL/irrelevance_31: a favorite restaurant recommendation asked; only a cell-division simulator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 248 OUT107

✓

NoToolForGlobalWarmingDocumentaryBFCL/irrelevance_38: a documentary (video) asked; only a scientific-research-paper retriever is available (wrong medium).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 249 OUT196

✓

NoToolForLasagnaIngredientsBFCL/irrelevance_47: recipe ingredients asked; only a flight-timetable tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 249 OUT239

✓

NoToolForLionDominantTraitBFCL/irrelevance_34: dominant genetic trait of a species asked; only a gene-variant-FREQUENCY lookup is available (different genetic output).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 251 OUT227

✓

NoToolForMultinucleatedCellNameBFCL/irrelevance_30: name of a cell type with multiple nuclei asked; only a blood-cell classifier (by shape/size) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 255 OUT88

✓

NoToolForPicnicSpotBFCL/irrelevance_37: a picnic spot asked; only a local-fauna information tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 248 OUT245

✓

NoToolForTranslationBFCL/irrelevance_52: natural-language translation asked; only a file-compression tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 256 OUT29

Tool Calling - Irrelevance: Motion, Electricity & Misc 10 passed

100.0%

IN35 428 OUT1 915

✓

NoToolForCarProjectileRangeBFCL/irrelevance_16: projectile-style "how far will a car travel" asked with symbolic v/t/theta; the range tool needs concrete numeric velocity + angle -> nothing to pass -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 553 OUT421

✓

NoToolForConeVolumeBFCL/irrelevance_19: cone volume asked; only an investment-return calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 547 OUT198

✓

NoToolForCurrentClockTimeBFCL/irrelevance_17: current clock time asked; only a travel-time (distance/speed) calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 536 OUT35

✓

NoToolForElectronMassBFCL/irrelevance_24: mass of an electron asked (a knowledge fact); only an electric-current tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 537 OUT46

✓

NoToolForFibonacciOf20BFCL/irrelevance_51: fibonacci number asked; only a cryptocurrency-price tool is available (the fibonacci calculator lives in a different file so it cannot satisfy this query).1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 538 OUT325

✓

NoToolForForceAngleFromMomentumBFCL/irrelevance_18: angle of a force from momentum asked; only a vector-angle-from-XY-components tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 543 OUT206

✓

NoToolForHorrorMovieListBFCL/irrelevance_50: movie recommendations asked; only a Celsius-to-Fahrenheit converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 543 OUT342

✓

NoToolForMagneticFieldNearWireBFCL/irrelevance_21: magnetic field near a current-carrying wire asked; only an EM-wave-amplitude tool is available (magnetic-field calculators are deliberately kept in a different file).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 552 OUT250

✓

NoToolForSineOf45DegreesBFCL/irrelevance_45: sine of an angle asked; only a polygon-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 538 OUT30

✓

NoToolForTeslaPriceBFCL/irrelevance_46: product price of a car asked; only a currency-exchange-rate tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 541 OUT62

Tool Calling - Irrelevance: Music Theory & Lookup 8 passed

100.0%

IN25 588 OUT1 136

✓

NoToolForAncientGreekLawmakersBFCL/irrelevance_171: who established laws in Ancient Greece asked (history); only a musical-piece composer search is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 196 OUT248

✓

NoToolForBasketballGameAtVenueBFCL/irrelevance_169: who is playing a basketball game at a venue asked; only concert-locating tools exist.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 199 OUT95

✓

NoToolForBluesScalePatternBFCL/irrelevance_176: the interval pattern of a blues scale asked; only composer lookups are available (get_blues_scale lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 201 OUT136

✓

NoToolForDonQuixoteAuthorBFCL/irrelevance_172: author of the novel Don Quixote asked; only music-composition lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 194 OUT54

✓

NoToolForFamousUsComposersBFCL/irrelevance_170: most famous US composers asked (general knowledge, no specific piece); only a chord-progression creator + piece-specific composer lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 197 OUT387

✓

NoToolForGrammyBestAlbumBFCL/irrelevance_177: Grammy Award winner asked; only a song chord-progression lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 202 OUT24

✓

NoToolForPrimaryTriadsOfKeyBFCL/irrelevance_173: primary triads of a key asked; only a genre-common-chords finder is available (primary_triads lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 200 OUT88

✓

NoToolForUniversityClassAvailabilityBFCL/irrelevance_168: availability of a university class asked; only concert-availability / concert-search tools are available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 199 OUT104

Tool Calling - Irrelevance: Probability & Legal 5 passed

100.0%

IN14 007 OUT890

✓

NoToolForFaceCardProbabilityBFCL/irrelevance_64: probability of drawing a face card from a deck asked; only a COIN-toss heads probability tool is available (coins, not cards) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 798 OUT88

✓

NoToolForHomeElectricityNeedBFCL/irrelevance_58: home electricity demand (kWh) asked; only a solar-panel COUNT calculator is available (it sizes panels, not electricity need, and needs sunlight hours not given) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN2 801 OUT194

✓

NoToolForNoiseComplaintProcessBFCL/irrelevance_91: how/whether to report a noise complaint asked; only a law-category listing tool is available (it lists legal categories, it does not file or advise on complaints) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 798 OUT188

✓

NoToolForRedMarbleCountBFCL/irrelevance_65: number of red marbles asked; only a determine-population tool is available (it returns a total population from probability + sample size, not a sub-category count) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN2 813 OUT76

✓

NoToolForTenantRightsBFCL/irrelevance_89: explaining tenant rights asked; only a contract-generation tool is available (generating a contract does not explain rights) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN2 797 OUT344

Tool Calling - Irrelevance: Recipes & Cooking 8 passed1 skipped

100.0%

IN27 658 OUT1 068

✓

NoToolForBestGameCharacterBFCL/irrelevance_206: best game character vs a dragon asked; only a best-WEAPON finder available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 461 OUT114

✓

NoToolForBestMovie2020BFCL/irrelevance_235: best MOVIE of 2020 asked; only a currency-exchange calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 458 OUT94

✓

NoToolForBuildingAPcBFCL/irrelevance_209: how to build a PC asked; only a cooking-recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 453 OUT404

✓

NoToolForFamousPlaceInParisBFCL/irrelevance_210: most famous PLACE in Paris asked; only a restaurant-by-dish search available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 455 OUT43

✓

NoToolForGameShortestPathBFCL/irrelevance_208: shortest in-game path asked; only a taxi-fare calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 463 OUT97

✓

NoToolForInchToMeterBFCL/irrelevance_234: LENGTH conversion (inches to metres) asked; only a CURRENCY converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 454 OUT60

✓

NoToolForLowestGameScoreBFCL/irrelevance_207: LOWEST game score asked; only a HIGHEST-score retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 458 OUT110

✓

NoToolForNewYorkPopulationBFCL/irrelevance_225: current POPULATION of New York asked; only a time-zone converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 456 OUT146

–

NoToolForQuickestFlightLondonTokyoBFCL/irrelevance_236: quickest way to fly London->Tokyo asked; a flight-duration tool is available. REVIEW FLAG (fairness): get_flight_duration plausibly answers this ("quickest ... by plane" ~= quickest flight duration), so a competent human might legitimately call it. Kept as BFCL labels it (irrelevant), but this item is borderline. Marked Hard because the intended "no call" is debatable.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_flight_duration' (with flight_type non-stop) plausibly answers 'quickest way ... by plane' - BFCL-mislabeled/borderline-relevant. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Irrelevance: Restaurants & Travel 8 passed

100.0%

IN26 531 OUT1 316

✓

NoToolForBeijingSunsetBFCL/irrelevance_227: SUNSET time asked; only a SUNRISE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 315 OUT56

✓

NoToolForClosestItalianRestaurantBFCL/irrelevance_216: FIND the closest Italian restaurant asked; only a currency converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 314 OUT53

✓

NoToolForFindSushiRestaurantBFCL/irrelevance_214: FIND a sushi restaurant asked; only a tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 316 OUT81

✓

NoToolForNearestPharmacyBFCL/irrelevance_237: FIND the nearest pharmacy asked; only a route-to-a-known-location planner available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 316 OUT54

✓

NoToolForPetFriendlyTrainStationBFCL/irrelevance_230: a pet-friendly TRAIN STATION asked; only hotel search tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 315 OUT177

✓

NoToolForSushiRestaurantsTokyoBFCL/irrelevance_229: FIND sushi RESTAURANTS asked; only hotel-booking tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 316 OUT265

✓

NoToolForThaiRestaurantChicagoBFCL/irrelevance_231: FIND a Thai RESTAURANT asked; only a hotel-availability checker available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 317 OUT74

✓

NoToolForYogaRetreatBFCL/irrelevance_232: a yoga RETREAT asked; only a HOTEL search tool available. REVIEW FLAG (fairness): a retreat is not a hotel, but hotel_search_find_hotels accepts arbitrary amenities (spa, vegetarian) and a location, so it is somewhat tempting. BFCL labels this irrelevant (retreat != hotel); kept as such and marked Hard because the boundary is borderline.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 322 OUT556

Tool Calling - Irrelevance: Sports & Games (set 1) 10 passed

100.0%

IN35 708 OUT1 422

✓

NoToolForAssistAverageBFCL/irrelevance_185: assist average asked; only an average-SCORING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 572 OUT58

✓

NoToolForBestChessMoveBFCL/irrelevance_194: best next move asked; only a board-game-STATISTICS tool is available (the move-suggesting board_analyser lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 574 OUT93

✓

NoToolForChessArrangementCountBFCL/irrelevance_196: total theoretical arrangements asked; only a final-score calculator is available (possible_moves lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 574 OUT302

✓

NoToolForFootballTeamRankingBFCL/irrelevance_186: ranking of a team asked; only a season-MVP tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 570 OUT159

✓

NoToolForIceCreamStoreBFCL/irrelevance_201: find a physical store asked; only a video-game guide is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 567 OUT38

✓

NoToolForMostHeartsInPokerBFCL/irrelevance_199: who holds most of a suit asked; only a poker-dealing tool is available (get_highest_card_holder lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 574 OUT123

✓

NoToolForPlayerBiographyBFCL/irrelevance_189: a biographical "who is X" asked; only a top-ranked-athlete lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 567 OUT172

✓

NoToolForTeamRosterBFCL/irrelevance_192: full roster asked; only a top-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 569 OUT109

✓

NoToolForUnoRulesBFCL/irrelevance_198: the rules of a game asked; only a deck-shuffling tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 571 OUT299

✓

NoToolForWorldCupWinnerBFCL/irrelevance_191: who won a tournament asked (no team named); only a per-team match-stats tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 570 OUT69

Tool Calling - Irrelevance: Sports & Games (set 2) 8 passed1 failed

88.9%

IN34 809 OUT979

✓

NoToolForBlackjackAceRuleBFCL/irrelevance_200: a rules question asked; only a poker highest-card-holder tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 469 OUT101

✓

NoToolForBookAuthorBFCL/irrelevance_205: author of a book asked; only a video-game score calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 473 OUT19

✓

NoToolForChessTournamentWinnerBFCL/irrelevance_195: who won a tournament asked; only a board-position move-suggester is available (the game-history fetch_game_stats lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 470 OUT111

✓

NoToolForCreateJigsawBFCL/irrelevance_204: CREATE a puzzle asked; only a puzzle-SOLVING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 463 OUT157

×

NoToolForHaloPlayerRankBFCL/irrelevance_203: a player's RANK asked; only a player-SCORE lookup is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN7 066 OUT340

✓

NoToolForLakersTopScorerBFCL/irrelevance_193: top scorer of a team asked; only a team-details tool (roster/results/upcoming, not scorers) is available (sports_team.get_top_scorer lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 468 OUT82

✓

NoToolForMonopolyWinnerBFCL/irrelevance_197: who won a casual game asked; only a possible-moves calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 468 OUT56

✓

NoToolForTopTennisPlayerBFCL/irrelevance_190: top ranked PLAYER asked; only a TEAM standing/ranking tool is available (the athlete-ranking get_top_ranked lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 467 OUT63

✓

NoToolForWorldSeriesWinnerBFCL/irrelevance_202: who won a real game asked; only a score calculator (from raw points) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 465 OUT50

Tool Calling - Irrelevance: Sports Statistics 8 passed1 failed

88.9%

IN35 128 OUT1 220

✓

NoToolForBasketballSeasonMvpBFCL/irrelevance_187: who won the MVP asked; only a team-ranking tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 511 OUT41

✓

NoToolForClippersRosterBFCL/irrelevance_179: roster of a team asked; only a two-team game-results tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 504 OUT179

✓

NoToolForCommonChordsInPopBFCL/irrelevance_174: most common chords in a pop song asked; only a key-signature primary-triads tool is available (find_common_chords lives in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 509 OUT396

×

NoToolForCricketMatchParticipantsBFCL/irrelevance_180: who is playing in cricket matches today asked; only a cricket-schedule tool (dates/matches, not participants) is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN7 058 OUT201

✓

NoToolForLaLigaPlayersBFCL/irrelevance_181: who played in a whole league asked; only a two-named-team last-match-result tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 504 OUT79

✓

NoToolForMoonlightSonataComposerBFCL/irrelevance_175: composer of Moonlight Sonata asked; only a blues-scale generator is available here (all composer-lookup tools are isolated in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 507 OUT59

✓

NoToolForNbaMvpWinnerBFCL/irrelevance_184: who won an MVP award asked (no player named); only a per-player stats tool that requires a player name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 513 OUT52

✓

NoToolForTopAssistPlayerBFCL/irrelevance_178: top ASSIST player asked; only a top-GOAL-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 508 OUT156

✓

NoToolForWimbledonChampionBFCL/irrelevance_183: winner of a tournament asked (no person named); only a celebrity-info tool that requires a name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 514 OUT57

Tool Calling - Irrelevance: Time, Weather & Misc 3 passed1 skipped

100.0%

IN7 742 OUT269

✓

NoToolForCityDistanceKmBFCL/irrelevance_239: DISTANCE (km) between two cities asked; only a get_date tool that returns the TIME difference between two locations is available (its 'unit' arg mentions distance, but it computes time) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 584 OUT64

–

NoToolForCurrentTimeSydneyBFCL/irrelevance_228: current time in Sydney asked; a local-time-by-city tool is available. REVIEW FLAG (fairness): get_local_time(location='Sydney') directly answers this, so a competent human would legitimately call it. BFCL nonetheless files it under irrelevance; kept as such to preserve the source label, but this item is effectively RELEVANT and should be reviewed. Marked Hard because the "no call" expectation is contradicted by the tool itself.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_local_time' directly answers 'current time in Sydney' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForTomatoCaloriesBFCL/irrelevance_222: CALORIES in a tomato asked; only a grocery-item-details lookup available. REVIEW FLAG (fairness): item_details at 'detailed' level could arguably surface nutrition info, and it requires a store_location the user never gave; BFCL labels it irrelevant (it is a grocery-store item lookup, not a nutrition-facts service). Kept as such; marked Medium as borderline.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 580 OUT61

✓

NoToolForWriteABookBFCL/irrelevance_217: "can you write a book" asked; only a recipe-CREATION tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN2 578 OUT144

Tool Calling - Irrelevance: Weather, Environment & Services 10 passed

100.0%

IN36 637 OUT1 843

✓

NoToolForBestCo2ReductionBFCL/irrelevance_108: the best way to reduce CO2 asked; only a numeric emission-reduction estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 662 OUT318

✓

NoToolForBestMonthToVisitBFCL/irrelevance_101: best month to visit Hawaii asked; only an average-monthly-temperature tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 660 OUT256

✓

NoToolForCactusNutrientsBFCL/irrelevance_109: a plant's nutrient needs asked; only a weekly watering-needs calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 667 OUT320

✓

NoToolForCurrentTimeWeatherBFCL/irrelevance_103: the current time asked; only a current-weather forecast tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 661 OUT46

✓

NoToolForGardenSpeciesCompositionBFCL/irrelevance_107: garden species composition asked; only a soil sand/silt/clay composition analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 664 OUT144

✓

NoToolForGmoYogaMatBFCL/irrelevance_111: a GMO yoga mat asked; only an eco-friendly product locator is available (GMO is not eco-friendly) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 663 OUT33

✓

NoToolForImpactfulCasesBFCL/irrelevance_94: a firm's impactful cases asked; only a single-case detail lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 671 OUT208

✓

NoToolForPaintingEiffelCaseResultBFCL/irrelevance_96: how long to paint the Eiffel Tower asked; only a law-case result lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 663 OUT92

✓

NoToolForPollinationFlowersBFCL/irrelevance_105: top pollination flower species asked; only a route elevation-profile plotter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 665 OUT221

✓

NoToolForTomatoFertilizerBFCL/irrelevance_106: best fertilizer for tomatoes asked; only a soil-type analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 661 OUT205

Tool Calling - Parallel Calls: Art, Music & Local Services 9 passed1 failed1 skipped

90.0%

IN80 178 OUT3 447

–

CalculateDrivingDistancesShortestRouteBFCL/parallel_38: two calc_distance calls — New York→Washington DC and Los Angeles→San Francisco. Tightened per methodology: the prompt explicitly enables "shortest route", so shortest_route == true is required (schema default is false).0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Skipped: Ambiguous scope (item flaw, not model behavior): the trailing 'with optional parameter shortest route enabled' can reasonably apply to only the second leg (its nearest clause) or to both; the BFCL ground truth requires it on both, but a competent human would not answer deterministically. Kept for traceability.

Skipped

IN0 OUT0

✓

CheckInstrumentAvailabilityInTwoCitiesBFCL/parallel_42: two check_instrument_availability calls — Yamaha P125 in Berlin and Madrid.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 490 OUT237

✓

FindMusicStoresInTwoCitiesBFCL/parallel_41: two find_music_instrument_store calls — San Francisco and Chicago; instruments {Yamaha acoustic guitar, Kawai piano} in any order and either casing.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 515 OUT317

✓

FindRockAndJazzConcertsInTwoCitiesBFCL/parallel_43: four concert_finder calls — {San Francisco, New York} x {rock, jazz} over the next month. time_period may be omitted (defaults to 30 days ≈ "next month"). Isolated from concert_find_nearby (parallel_44) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 810 OUT456

✓

GetExhibitionDetailsForTwoMuseumsBFCL/parallel_40: two museum_exhibition_detail calls — 'Wonder of Nature' at the Louvre and 'Age of Reptiles' at the British Museum. visitor_type {child, adult} (unordered). Tightened per methodology: the prompt explicitly asks for cost "for children and adult", so both visitor types are required (schema default is ['adult'] only).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 552 OUT377

✓

GetMostPlayedPopAndRockSongsBFCL/parallel_45: two musicCharts_getMostPlayed calls — most played Pop and Rock song in Australia. Optional duration omitted (ground truth accepts the schema default of 0).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 474 OUT147

✓

GetMuseumInfoForTwoCitiesBFCL/parallel_39: two museum_info_get_info calls — Washington D.C. and Paris; details {Opening hours, Adult tickets, Child tickets} in any order.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 504 OUT351

×

GetRgbAndHexForThreeColorsBFCL/parallel_37: three color_converter_get_color_info calls — navy, purple, maroon; each with {RGB, HEX} (unordered).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN12 064 OUT508

✓

GetTeamRankingsInTwoLeaguesBFCL/parallel_47: two get_team_ranking calls — Barcelona in UEFA Champions League and Manchester United in La Liga.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 475 OUT296

✓

MixPaintColorsForTwoRequestsBFCL/parallel_36: two paint_color_mixture calls — watercolor magenta and acrylic navy.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 524 OUT319

✓

WinningPercentageForTwoTeamsTwoSeasonsBFCL/parallel_46: four calculate_winning_percentage calls — {Lakers, Bulls} x {2018, 2020}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 770 OUT439

Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 1) 12 passed1 failed

92.3%

IN116 714 OUT5 091

✓

AverageFourNumberSetsBFCL/parallel_132: four calculate_average calls — one per list of numbers (element order preserved).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 768 OUT362

✓

CalculateBmiFourPeopleMetricBFCL/parallel_134: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt states "All measurements are in the metric system", so unit == "metric" is required (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 794 OUT455

×

CalculateCityDistancesInKilometersBFCL/parallel_135: three geo_distance_calculate calls — NY->LA->Miami->NY, in kilometers. Tightened: the tool defaults to miles but the prompt asks "in kilometers", so units == "kilometers" is REQUIRED (omission would return miles, contradicting the request — the ground truth's "omit" option is unsafe here).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN13 451 OUT679

✓

CalculateGpsDistancesAlongRouteBFCL/parallel_133: three calculate_distance calls over GPS coordinate pairs (Paris->Rome->Athens-> Giza), each in kilometers. Uses the coordinate-based tool (coord1/coord2), not the city-name distance tools that are also visible.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 791 OUT426

✓

CompareAverageTemperatureNyAndLaBFCL/parallel_128: two average_temperature calls — New York 7 days in Fahrenheit, Los Angeles 7 days in Celsius. Tightened: the prompt explicitly requests Fahrenheit for NY, so temp_unit == "Fahrenheit" is required (also the default); Celsius is non-default and required for LA.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 362 OUT198

✓

CreateTwoHistogramsWithFiveBinsBFCL/parallel_129: two create_histogram calls — one per data set, 5 bins each (data order preserved).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 826 OUT521

✓

EstimateElephantPopulationThreeScenariosBFCL/parallel_125: three elephant_population_estimate calls — current 500 at 2%, 1.5%, 2.5% over 10y.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 703 OUT375

✓

FindItalianRestaurantsInNyAndLaBFCL/parallel_130: two find_restaurants calls — New York and Los Angeles, Italian, 4 results each, dietary_requirements [vegan, gluten-free]. Distinct from restaurant_find_nearby (this tool carries food_type + number, which the prompt supplies, so it is the correct choice here).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 426 OUT282

✓

FindRestaurantsByDietInThreeCitiesBFCL/parallel_127: three restaurant_find_nearby calls — New York, Los Angeles, Chicago, each with dietary_preference [Vegan, Gluten-free, Dairy-free]. List order follows the prompt (matches ground truth).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 549 OUT328

✓

PredictEvolutionDarwinAndLamarckBFCL/parallel_126: two prediction_evolution calls — African Elephant over 5000y, Darwin then Lamarck. Tightened per methodology: the prompt explicitly says "using the Darwin model", so model == "Darwin" is required for the first call (ground truth also accepts omission via the 'Darwin' default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 385 OUT275

✓

ProjectPopulationGrowthJapanAndIndiaBFCL/parallel_124: four population_projections calls — Japan/10y and India/20y, each at the current (default) growth rate and at an explicit rate. "current growth rate" maps to the default (growth_rate omitted, accepts null or 0.01); the explicit 1.5%/2.1% rates are required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 710 OUT465

✓

RouteThreeLegsAvoidingTollsBFCL/parallel_131: three map_routing_fastest_route calls — SF->Palo Alto->San Jose->home, all avoiding tolls. Tightened: the prompt explicitly says "avoiding toll roads", so avoid_tolls == true is required for every leg (non-default value stated in the prompt).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 555 OUT284

✓

ShortestCityDistanceBusWithAndWithoutTransferBFCL/parallel_136: two city_distance_find_shortest calls — NY->LA by bus, first without transfer then allowing transfer. Tightened: the prompt explicitly says "using a bus", so transportation == "bus" is required for both (also the default). allow_transfer: false/omitted for the first, true for the second.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 394 OUT441

Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 2) 14 passed

100.0%

IN125 846 OUT4 659

✓

CalculateAreaOfThreeTrianglesBFCL/parallel_143: three calc_area_triangle calls — (10,5), (15,7), (20,10).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 962 OUT298

✓

CalculateBinomialProbabilityBasketballBFCL/parallel_161: three calculate_binomial_probability calls — basketball shooter at 60%: (10,7), (15,10), (20,15). probability_of_success == 0.6 is required (prompt states 60%). ISOLATED here (away from prob_dist.binomial in ToolCalling-Parallel-D3): both tools compute binomial probability, and this prompt does not name a function, so having both visible would make the choice genuinely ambiguous. Kept apart so exactly one correct tool is available.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 041 OUT375

✓

CalculateBmiInMetersForThreePeopleBFCL/parallel_138: three calculate_BMI calls (weight kg, height meters) — John, Sarah, Mike.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 023 OUT341

✓

CalculateCelestialDistancesInMilesBFCL/parallel_149: two calculate_distance calls between celestial bodies (Mars-Venus, Mars-Jupiter), in miles. Uses the body-name distance tool (body1/body2). Tightened: prompt asks "in miles", so unit == "miles" is required (tool default is kilometers, so omission would be wrong).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 768 OUT249

✓

CalculateClockAnglesAtThreeTimesBFCL/parallel_145: three calculate_clock_angle calls — 3:15, 8:20, 11:50. Tightened: the prompt asks for results "rounded to 2 decimal places", so round_to == 2 is required (also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 943 OUT359

✓

CalculateFactorialOfFourNumbersBFCL/parallel_144: four math_factorial calls — 5, 3, 4, 2.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 972 OUT237

✓

CalculateLightTravelTimeThreeLegsBFCL/parallel_147: three light_travel_time calls — 4.22, 6.1, 5.88 light years. Tightened: the prompt explicitly states the speed of light (299792458 m/s), so speed_of_light == 299792458 is required (also default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 029 OUT385

✓

CalculateSpeedForCarBicycleTrainBFCL/parallel_148: three calculate_speed calls — car 500m/25s in km/h, bicycle 1000m/200s in m/s, train 10000m/600s in km/h. Each output unit is explicitly named in the prompt, so to_unit is required.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 044 OUT305

✓

FetchAllDataFieldsForTwoEmployeesBFCL/parallel_139: two employee_fetch_data calls — Tech Solutions, employees 12345 and 67890, each fetching all four data fields (order follows the prompt / ground truth).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 876 OUT270

✓

FindDiCaprioDramaAndComedyMoviesBFCL/parallel_140: two imdb_find_movies_by_actor calls — Leonardo DiCaprio, Drama in 2010 and Comedy in 2012. Tightened: the prompt explicitly names each category, so category is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 764 OUT244

✓

GetImaxAndTwoDTheaterReleasesBFCL/parallel_141: two get_theater_movie_releases calls — New York IMAX over 7 days, Los Angeles 2D over 14 days. Tightened: the prompt explicitly names each format, so format is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 793 OUT268

✓

PlotTwoSineWavesBFCL/parallel_146: two plot_sine_wave calls with all five parameters supplied by the prompt.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 899 OUT295

✓

SortFourListsAscendingAndDescendingBFCL/parallel_137: eight array_sort calls — four lists, each sorted ascending AND descending (list element order preserved as given).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 848 OUT772

✓

UpdateTwoCustomersNameAndEmailBFCL/parallel_142: two update_user_info calls — users 12345 and 67890, both set name "John" and email "example@.com". update_info is a nested object. Tightened: the prompt names the 'CustomerInfo' database, so database == "CustomerInfo" is required (also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 884 OUT261

Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 3) 11 passed

100.0%

IN87 979 OUT4 204

✓

AreaUnderCurveForTwoPolynomialsBFCL/parallel_150: two mathematics_calculate_area_under_curve calls — polynomial [3,-2,1] over [-1,2] and [1,0,-1] over [0,3] (coefficient/limit order preserved).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 834 OUT246

✓

BinomialProbabilitiesThreeGamesBFCL/parallel_160: three prob_dist_binomial calls — (20,5), (50,15), (100,30), each with p == 0.3 (prompt states a 0.3 win probability). The prompt explicitly names 'prob_dist.binomial'; the colliding calculate_binomial_probability tool is kept in a separate script (ToolCalling-Parallel-D2).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 005 OUT656

✓

CalculateBmiSystemFourPeopleBFCL/parallel_154: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt says "use the metric system for all calculations", so system == "metric" is required (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 132 OUT459

✓

DiceRollProbabilitiesThreeScenariosBFCL/parallel_159: three probability_dice_roll calls — roll 4 three times and roll 2 twice on a six-sided die (die_sides default 6, accepts omission), then roll 7 twice on an eight-sided die (die_sides == 8 required, non-default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 962 OUT396

✓

GenerateNormalVariatesTwoConfigsBFCL/parallel_158: four random_normalvariate calls — two at (mu 5, sigma 2) and two at (mu 10, sigma 3). Duplicate calls: each configuration must appear exactly twice (multiplicity 2).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 068 OUT357

✓

PredictPricesForFourHousesBFCL/parallel_157: four predict_house_price calls — New York (3/2/1500), Los Angeles (4/3/2000), Chicago (2/1/1200), Miami (3/2/1800).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 256 OUT510

✓

RaiseTwoNumbersToPowersBFCL/parallel_152: two math_power calls — 3^5 and 2^3, no modulus (mod omitted / null).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 688 OUT136

✓

RunLinearRegressionWithAndWithoutStandardizationBFCL/parallel_155: two run_linear_regression calls — same predictors [Age, Income, Education] and target "Spending Score", first without standardization then with. standardize is explicitly requested (false then true), so it is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 880 OUT265

✓

TrainRandomForestFourConfigsBFCL/parallel_156: four random_forest_train calls on "data_random_forest" — (100,10), (200,20), (300,30), (400,40). The prompt explicitly names this function, disambiguating it from train_random_forest_classifier visible in this script.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 314 OUT519

✓

TrainRfClassifierTwoConfigsBFCL/parallel_153: two train_random_forest_classifier calls — dataset1 (depth 10, 100 trees) and dataset2 (depth 20, 200 trees). The prompt explicitly names this function, disambiguating it from the other random-forest training tool (random_forest_train) visible in this script.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 878 OUT260

✓

TriangleAreasInDifferentUnitsBFCL/parallel_151: three geometry_area_triangle calls in different units — (15,20) square meters, (25,30) square feet, (35,40) square inches. First unit is the default (accepts omission / square meters); feet and inches are non-default and required.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 962 OUT400

Tool Calling - Parallel Calls: Finance, Stats & Legal 13 passed

100.0%

IN110 804 OUT3 922

✓

BuildTwoDecisionTreeClassifiersBFCL/parallel_12: two model_DecisionTreeClassifier calls — gini/depth 5 and entropy/depth 10, both random_state 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 444 OUT234

✓

CalculateFutureValueForThreeTermsBFCL/parallel_18: three financials_calculate_future_value calls — years 1, 5, 10 (same 1000 @ 0.05).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 606 OUT337

✓

CalculatePresentValueForThreeTermsBFCL/parallel_14: three calculate_present_value calls — years 10, 20, 30 (same payment 1000, rate 0.05).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 603 OUT295

✓

CapitalGainsTaxForTwoStatesBFCL/parallel_15: two calculate_capital_gains_tax calls — CA (15000/25000) and FL (20000/50000).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 496 OUT303

✓

ComputePythagorasForTwoTrianglesBFCL/parallel_10: two math_pythagoras calls — sides (3,4) and (5,12).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 387 OUT192

✓

ConfidenceIntervalForTwoSampleSizesBFCL/parallel_13: two confidence_interval_calculate calls — sample sizes 50 and 150. Tightened per methodology: the prompt explicitly requests "95% confidence interval", so confidence_level == 0.95 is required (schema default is 0.9, but the prompt states the value).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 499 OUT475

✓

CourtCaseStatusAndTrialDatesBFCL/parallel_22: four court_info_get_case_status calls — cases XY1234 (LA) and GH5678 (Orange County), each queried for status (details may be omitted via the 'status' default) and for trial_date.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 758 OUT369

✓

GetStockPriceAndVolumeForThreeCompaniesBFCL/parallel_17: three get_stock_data calls — AAPL, GOOG/GOOGL, MSFT; each with data_points {price, volume} (unordered).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 575 OUT435

✓

LoanEligibilityForTwoInstitutionsBFCL/parallel_20: two loan_eligibility_check calls — HSBC (500k/100k) and Wells Fargo (700k/120k).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 511 OUT250

✓

MortgagePaymentForThreeLoanTermsBFCL/parallel_19: three calculate_mortgage_payment calls — loan terms 15, 20, 30 (400000 @ 0.04).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 631 OUT307

✓

PredictHousePriceInTwoCitiesBFCL/parallel_11: two ml_predict_house_price calls — New York 3000 sq ft, Los Angeles 4000 sq ft.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 418 OUT245

✓

ReturnOnInvestmentGainAndLossBFCL/parallel_16: two calculate_return_on_investment calls — gain 500 on 2000, loss -1000 on 5000.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 449 OUT202

✓

SearchMoneyLaunderingConvictionsBFCL/parallel_21: two law_crimes_search calls — money laundering in San Francisco 2019 and Texas 2018.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 427 OUT278

Tool Calling - Parallel Calls: Legal, History & Environment 12 passed1 failed1 skipped

92.3%

IN123 287 OUT4 780

✓

BookTwoConnectingFlightsBFCL/parallel_30: two book_flight calls — San Francisco→Tokyo (2022-05-03) and Tokyo→Sydney (2022-05-18).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 061 OUT212

✓

CalculateAlimonyForTwoDurationsBFCL/parallel_23: two alimony_calculator_ca_calculate calls — durations 10 and 20 years (10000/3000 income).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 124 OUT431

✓

CalculateWasteForFamilyAndBachelorBFCL/parallel_29: two waste_calculation_calculate calls — LA family (2 adults, 2 children) and New York bachelor (1 single). The population argument is a nested object.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 074 OUT245

✓

FetchTwoHistoricalFactsBFCL/parallel_31: two history_fact_fetch calls — Treaty of Paris and Magna Carta. Optional depth/year omitted (ground truth accepts the schema defaults).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 980 OUT414

✓

FindClassicalConcertsWithParkingBFCL/parallel_44: two concert_find_nearby calls — classical concerts near Berlin and Paris next Friday. Tightened per methodology: the prompt explicitly asks for concerts "with available parking", so amenities == ['Parking'] is required (this also matches the schema default, but the prompt states it). Isolated from concert_finder (parallel_43) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 038 OUT195

–

FindCourtCasesByNumberAndTypeBFCL/parallel_26: four court_case_find calls — the cross product of case numbers {67813, 71249} and types {Civil, Criminal}; case_number is a single-element array; Civil may be omitted via its default. FLAGGED as borderline (see report): the prompt "case numbers '67813','71249' for type 'Civil' and 'Criminal'" is ambiguous about the pairing; BFCL ground truth expects all four combinations, kept faithful.0 AI queries 0 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Skipped: Ambiguous call decomposition (item flaw, not model behavior): 'case numbers 67813, 71249 for type Civil and Criminal' has no single deterministic reading - two calls (both numbers per type, case_number as a list) is as valid as the BFCL 2x2 = four single-number calls; a competent human would not answer deterministically. Kept for traceability.

Skipped

IN0 OUT0

✓

FindNatureReservesNearTwoCitiesBFCL/parallel_27: two nature_reserve_find_nearby calls — Berkeley (10km, picnic tables + restrooms) and Tokyo (5km, playgrounds + biking trails). Amenity list order is unordered.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 062 OUT225

✓

GetCaseDetailsWithHistoryAndLitigantsBFCL/parallel_24: two law_case_get_details calls — cases 28473 and 64725. Tightened per methodology: the prompt explicitly asks for "their history and details of litigants", so include_history == true and include_litigants == true are required (schema defaults are false).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 044 OUT282

✓

GetEventsForTwoPresidenciesBFCL/parallel_32: two us_history_events_by_presidency calls — Abraham Lincoln and George Washington. Optional start_year/end_year omitted (ground truth accepts the schema defaults).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 975 OUT538

×

GetPresidentsAndVicePresidentsByYearBFCL/parallel_33: four get_president_and_vp calls — president for 1980 and 2016, vice president for 1975 and 2011.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN14 837 OUT620

✓

GetReligionInfoForTwoEmpiresBFCL/parallel_35: two ancient_empires_get_religion_info calls — Persian Empire and Mauryan Empire. Tightened per methodology: the prompt explicitly asks for "religious history and influences", so include_influences == true is required (schema default is false).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 015 OUT263

✓

GetTemperatureForTwoCitiesBFCL/parallel_28: two get_current_and_future_temperature calls — Seattle and Los Angeles, next 3 hours.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 987 OUT170

✓

LookupLawsuitsForTwoYearsBFCL/parallel_25: two lawsuit_lookup calls — Dara Inc for years 2019 and 2018.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 032 OUT165

✓

TrackChristianityInTwoRegionsBFCL/parallel_34: two religion_history_track calls — Christianity in Egypt and Turkey, 100–1500 A.D.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 058 OUT1 020

Tool Calling - Parallel Calls: Markets & Environment 13 passed

100.0%

IN114 815 OUT5 328

✓

CalculateCagrForTwoInvestmentsBFCL/parallel_176: two calculate_cagr calls — (5000,7000,5) and (8000,12000,3).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 669 OUT310

✓

CalculateEmissionSavingsForSolarAndWindBFCL/parallel_186: two calculate_emission_savings calls — solar(12 months) and wind(8 months), Midwest region. Prompt states the Midwest region explicitly -> region required (accept "Midwest"/"Midwest region").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 562 OUT294

✓

CompareFutureValueWithAndWithoutContributionsBFCL/parallel_182: two finance_calculate_future_value calls — (5000,0.07,10, no contribution) and (3000,0.06,10, contribution 200). First scenario states "no additional contributions" -> null or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 729 OUT253

✓

ComputeEuclideanNormForThreePointsBFCL/parallel_183: three math_hypot calls — (5,7), (10,15), (20,25). z omitted or 0 (2D points).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 765 OUT383

✓

EstimatePopulationsForTwoSpeciesAcrossTwoYearsBFCL/parallel_185: four estimate_population calls — Bengal Tigers/India and African Elephants/Kenya for 2021 and for 2023. NOTE: upstream ground truth's third entry lists year "" (omitted -> default 2020), which is an upstream error: the prompt explicitly asks for 2023 for "both these species". We tighten both 2023 estimates to year == 2023 (fair deterministic answer).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 908 OUT627

✓

FindRootsForTwoQuadraticEquationsBFCL/parallel_184: two algebra_quadratic_roots calls — (3,7,2) and (5,-4,1).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 682 OUT340

✓

GetClosingPricesForTwoCompaniesOnTwoDatesBFCL/parallel_178: four get_stock_price (by-date variant) calls — the 2x2 cross product of {Microsoft, Apple} x {2022-01-01, 2022-02-01} on NASDAQ. Upstream ground truth's first entry loosely lists company_name as [Microsoft, Apple]; the deterministic human answer is the four distinct (company, date) combinations, which we require. exchange NASDAQ (default) may be omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 940 OUT599

✓

GetDetailedAirQualityForThreeDaysBFCL/parallel_187: three get_air_quality calls — New York City, detail true, for 2023-05-05, 05-04, 05-03. Prompt asks for PM2.5/PM10/ozone/sources -> detail == true; "today/yesterday/day before" with today = May 5, 2023.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 768 OUT370

✓

GetFourPriceTypesForTwoCompaniesBFCL/parallel_180: eight stock_price calls — {Microsoft, Apple} x {Open, Close, High, Low}, days 30. Prompt explicitly lists all four data types, so data_type must be set explicitly (required) to distinguish the eight calls; omission (default 'Close') cannot represent the four requested types.1 AI query 8 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Passed

IN9 490 OUT775

✓

GetPricePerOunceForFourMetalsBFCL/parallel_177: four get_metal_price calls — gold, silver, platinum, palladium, measure 'ounce'. Prompt asks "price per ounce" -> measure == "ounce" required.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 789 OUT340

✓

GetStockPricesForFourCompaniesAndDurationsBFCL/parallel_181: four get_stock_prices calls — Apple(1 week), Microsoft(2 weeks), Amazon(3 weeks), Tesla(1 month). Each 'companies' argument is a single-element list.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 838 OUT282

✓

GetTrafficInfoForThreeLegsWithModesBFCL/parallel_188: three get_traffic_info calls — NY->LA (driving), LA->SF (bicycling), SF->NY (transit). First leg explicitly says "driving" -> mode == "driving" required (tightened; also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 763 OUT301

✓

ProjectFutureValueForThreeStocksBFCL/parallel_175: three portfolio_future_value calls — AAPL(5000,0.07,5), MSFT(8000,0.06,7), AMZN(10000,0.08,10).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 912 OUT454

Tool Calling - Parallel Calls: Mathematics & Physics (set 1) 13 passed

100.0%

IN104 090 OUT3 829

✓

AreaOfTwoRightTrianglesInMetersBFCL/parallel_68: two calculate_area calls — right triangles base/height 12x15 and 18x24 in meters. unit required (tool default is 'cm'; the prompt states meters/square meters).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 994 OUT281

✓

CheckPlayerStatusAcrossThreeSeasonsBFCL/parallel_49: three player_status_check calls — player 3142 in RocketLeague for seasons 2017/2018/2019. Seasons are explicitly given, so each is required (default 2023 not applicable).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 075 OUT256

✓

CircumferenceOfFourCirclesInCmBFCL/parallel_65: four calculate_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt gives every radius in cm (e.g. "5cm") and the tool default unit is 'm', so a correct call must pass unit == 'cm'/'centimeter' (ground truth requires it on the first call and merely allows omission on the rest; we require it consistently since it is stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 217 OUT401

✓

ConvertEuroAndPoundsToDollarsBFCL/parallel_59: two get_conversion_cost calls — 15000 Euro->dollars and 200 pounds->dollars.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 907 OUT303

✓

FinalVelocityForFourDropsBFCL/parallel_81: four calculate_final_velocity calls — heights 10/20/15/25, initial_velocity 0 (stated), gravity 9.81 (default, may be omitted).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 402 OUT440

✓

FindItalianAndJapaneseRestaurantsBFCL/parallel_52: two restaurant_finder calls — Italian in New York (preferences ['Vegetarian']) and Japanese in Los Angeles (delivery; ['Delivery'] or omitted since default is ['Delivery']).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 888 OUT220

✓

FindRootsOfTwoQuadraticsAlgebraBFCL/parallel_62: two algebra_quadratic_roots calls — 3x^2+4x+2 and 5x^2-7x+3.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 054 OUT272

✓

FindSoupAndVeganSaladRecipesBFCL/parallel_51: two recipe_search_find calls — Chicken Noodle Soup (diet omitted/default 'Keto') and a Vegan Salad (diet == 'Vegan', dish name accepts several phrasings).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 835 OUT174

✓

GcdOfJohnAndMaryNumbersBFCL/parallel_80: two math_gcd calls — gcd(36,48) and gcd(60,96). Same tool as parallel_76.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 950 OUT300

✓

GcdOfTwoPairsBFCL/parallel_76: two math_gcd calls — gcd(45,60) and gcd(81,27).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 882 OUT154

✓

GetPokemonMovesAndCheckSolarBeamBFCL/parallel_48: two PokemonGO_get_moves calls — Pikachu's move list (move omitted/default 'Run') and a check whether Bulbasaur can learn 'Solar Beam'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 872 OUT296

✓

PrimeFactorsFormattedAndArrayBFCL/parallel_74: four get_prime_factors calls — numbers 4567 and 7890, each formatted true and false (user wants "a formatted string as well as an array").1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 164 OUT598

✓

SaveGameProgressEasyAndHardBFCL/parallel_50: two game_save_progress calls — stage 7 easy and stage 3 hard; level omitted/default 'user'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 850 OUT134

Tool Calling - Parallel Calls: Mathematics & Physics (set 2) 13 passed

100.0%

IN107 973 OUT4 846

✓

AreaOfThreeCirclesInMetersBFCL/parallel_66: three geometry_area_circle calls — radii 5/10/15 in meters (default is meters, so units may be omitted).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 339 OUT379

✓

BookTwoHotelRoomsBFCL/parallel_57: two hotel_booking_book calls — Sheraton (NY, 2 adults/1 child) and Marriott (LA, 1 adult/2 children).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 387 OUT364

✓

CheckAvailabilityForTwoHotelsBFCL/parallel_56: two hotel_booking_check_availability calls — Queens Hotel (Berlin) and Royal Hotel (Paris), each with check-in/out dates (several date formats accepted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 319 OUT295

✓

CircumferenceOfFourCirclesInMetersBFCL/parallel_70: four geometry_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt asks for output in meters and gives radii in m, but the tool default unit is 'cm', so a correct call must pass units == 'm'/'meters' (ground truth requires it on the first call and only allows omission on the rest; we require it consistently since it is explicitly stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 519 OUT471

✓

DerivativeOfPolynomialAndItsDerivativeBFCL/parallel_71: two calculate_derivative calls — 3x^3-2x^2+5x-7 at x=4, then its derivative 9x^2-4x+5 at x=2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 234 OUT377

✓

FinalVelocityForThreeDragRaceRoundsBFCL/parallel_83: three final_velocity calls — (v0=0,a=5,t=10), (v0=10,a=7,t=8), (v0=20,a=4,t=12).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 516 OUT465

✓

FindGroceryStoresByCriteriaBFCL/parallel_55: two grocery_store_find_by_criteria calls — NYC ['24 hours'] and San Diego ['Home Delivery'].1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 152 OUT147

✓

GetTwoCookingRecipesBFCL/parallel_53: two get_cooking_recipe calls — Lasagne Bolognese for 4 and Caesar Salad for 2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 170 OUT298

✓

GetTwoExchangeRatesBFCL/parallel_58: two get_exchange_rate calls — USD->AUD and USD->CAD.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 134 OUT221

✓

HighestCommonFactorOfFourPairsBFCL/parallel_77: four math_hcf calls — (45,60), (90,120), (36,48), (72,96). FLAGGED (see final report): the prompt is phrased as chaining ("use that result to find the HCF with...") but the ground truth treats it as four INDEPENDENT parallel calls on the literal pairs; a human might interpret it as sequential composition. Kept BFCL-faithful (four independent literal-pair calls) since parallel semantics preclude feeding one call's result into another.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 553 OUT1 014

✓

OrderSameItemsFromTwoWholeFoodsLocationsBFCL/parallel_54: two whole_foods_order calls — same two items (order-independent) at downtown and uptown, size large.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 200 OUT215

✓

PrimeFactorizeAsDictionaryBFCL/parallel_79: two prime_factorize calls — 4567 and 7890, return_type 'dictionary' (explicitly requested).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 176 OUT371

✓

SolveTwoQuadraticEquationsBFCL/parallel_63: two solve_quadratic_equation calls — (5,6,1) and (3,2,1). Prompt names this function explicitly.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 274 OUT229

Tool Calling - Parallel Calls: Mathematics & Physics (set 3) 12 passed

100.0%

IN98 303 OUT4 071

✓

AreaOfTwoCirclesInMetersBFCL/parallel_67: two geometry_calculate_area_circle calls — radii 5 and 10 in meters. unit required (tool default is 'cm'; the prompt states meters).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 023 OUT288

✓

AreaOfTwoTrianglesBFCL/parallel_69: two calculate_triangle_area calls — base/height 10x5 and 8x6.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 083 OUT199

✓

DerivativeOfTwoFunctionsAtValuesBFCL/parallel_73: two calculus_derivative calls — 3x^2+2x-1 at 5 (variable x, default) and 4y^3-3y^2+2y-1 at 3 (variable y).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 128 OUT359

✓

DisplacementForThreeCarsBFCL/parallel_84: three calculate_displacement calls — (15,7,accel 3.5), (20,10,accel 2.0), (25,8,no acceleration). The third car travels "without any acceleration" so acceleration == 0 (also the default; omission accepted).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 344 OUT322

✓

EuclideanNormForThreePointsBFCL/parallel_61: three math_hypot calls — (3,4) and (6,8) with z omitted/0, and (9,12,15).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 258 OUT423

✓

FactorialOfThreeNumbersBFCL/parallel_60: three math_factorial calls — 5, 7, 9.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 085 OUT180

✓

FinalSpeedForTwoFreeFallObjectsBFCL/parallel_85: two calculate_final_speed calls — initial_speed 0 for 10 s and initial_speed 5 for 7 s; gravity -9.81 (default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 151 OUT450

✓

GcdViaNumberTheoryBFCL/parallel_78: two number_theory_gcd calls — (45,60) and (81,63). Prompt names this function explicitly.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 026 OUT155

✓

IntegrateTwoFunctionsWithTwoMethodsBFCL/parallel_72: four integrate calls — x**3 on [2,5] and 2x**2+3x-1 on [-1,3], each with trapezoid (default, may be omitted) and simpson (explicit).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 476 OUT487

✓

PrimeFactorsOfThreeNumbersBFCL/parallel_75: three number_analysis_prime_factors calls — 45, 100, 150.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 091 OUT250

✓

SolveFourQuadraticsWithRootTypeBFCL/parallel_64: four solve_quadratic calls — (2,5,3) all roots, (1,-3,2) real, (4,-7,3) all roots, (1,2,1) real. 'all' is the default so it may be omitted; 'real' is explicitly requested and required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 542 OUT776

✓

VelocityForTwoCyclingDaysBFCL/parallel_82: two calculate_velocity calls — 120 km in 5 h and 150 km in 6 h; unit km/h (default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 096 OUT182

Tool Calling - Parallel Calls: Mixed Domains 9 passed1 failed

90.0%

IN73 573 OUT2 593

✓

CalculateBmiForTwoPeopleBFCL/parallel_4: two calculate_bmi calls — 6 ft / 80 kg and 5.6 ft / 60 kg.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 282 OUT288

✓

CalculateEmForceForTwoTimeIntervalsBFCL/parallel_1: two calculate_em_force calls — same field/area, d_time 4 then 10.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 303 OUT218

✓

CalculateFactorialForThreeNumbersBFCL/parallel_7: three math_factorial calls — 5, 10, 15.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 342 OUT205

✓

CalculateResistanceForCopperAndAluminumBFCL/parallel_2: two calculate_resistance calls — copper then aluminum, same length/area. Tightened per methodology: the prompt explicitly names "resistivity of copper", so the first call must pass resistivity == "copper" (ground truth also accepts omission via the 'copper' default, but an explicitly-stated value must be supplied). Aluminum is non-default → required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 290 OUT373

✓

CalculateSalesTaxForThreeCitiesBFCL/parallel_6: three calculate_sales_tax calls — Chicago/IL $30.45, Sacramento/CA $52.33, Portland/OR $11.23.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 474 OUT290

✓

FetchPopulationForCitiesStateAndCountryBFCL/parallel_8: four database_us_census_get_population calls — NYC (city), LA (city), Alaska (state), USA (country); year defaults to 2000 (may be omitted). Requires area-type inference.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 558 OUT261

×

FindMovieShowingsForTwoMoviesBFCL/parallel_9: two find_movie_showing calls near San Diego — Tenet at 5 pm and No Time To Die at 7:30 pm.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: match0 == 1 Values: match0 = 0

Failed

IN7 296 OUT253

✓

GetSequenceAnd3DForThreeProteinsBFCL/parallel_3: three protein_info_get_sequence_and_3D calls — HbA1c, normal + rat hemoglobin. Tightened per methodology: the prompt explicitly asks for "their 3D models", so model_3d == true is required (ground truth also accepts omission via the default of true).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 421 OUT252

✓

ListShowRatingsOnNetflixAndHuluBFCL/parallel_5: two streaming_services calls — Netflix['Friends'] and Hulu['The Office','Stranger Things'] (Hulu show order unordered), both sorted by rating. Must group shows per service.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 341 OUT313

✓

PlayTwoArtistsForGivenDurationsBFCL/parallel_0: two spotify_play calls — Taylor Swift for 20 min and Maroon 5 for 15 min.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 266 OUT140

Tool Calling - Parallel Calls: Physical & Life Sciences (set 1) 12 passed

100.0%

IN102 090 OUT3 981

✓

ElectricFieldForTwoChargesBFCL/parallel_94: two calculate_electric_field calls — charge 5/distance 2 and charge 3/distance 4. "Conducted in a vacuum" = the default permitivity → may be omitted or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 414 OUT195

✓

ElectromagneticForceInTwoMediaBFCL/parallel_96: two electromagnetic_force calls — same charges 5,10 and distance 2, differing by medium_permittivity. Vacuum call (8.854e-12) equals the default → may be omitted or 8.854e-12; the second medium (5e-12) is non-default → required.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 570 OUT282

✓

EstimateTravelTimeForThreeRoutesBFCL/parallel_93: three route_estimate_time calls — (1) NYC->Miami via Philadelphia/Washington D.C./Atlanta, (2) NYC->Miami skipping Atlanta (via Philadelphia/Washington D.C.), (3) Philadelphia-> Miami via Washington D.C. Stop orderings from ground truth accepted. FLAG parallel_93: BORDERLINE. Ground truth allows many stop-list variants, some including the start city itself as a stop; a competent human's stop set is not fully deterministic. Kept BFCL-faithful by accepting the natural stop orderings (start city excluded from stops).1 AI query 3 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Passed

IN8 565 OUT320

✓

FinalSpeedForTwoDroppedObjectsBFCL/parallel_88: two calculate_final_speed calls — v0 0/height 10 and v0 5/height 20. gravity stated as 9.8 (the default) so it may be omitted or 9.8 per ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 420 OUT277

✓

FinalVelocityForTwoCarsBFCL/parallel_87: two calculate_final_velocity calls — Car A (v0 0, a 6, t 10), Car B (v0 20, a 4, t 15).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 437 OUT236

✓

FinalVelocityFromDistanceForTwoObjectsBFCL/parallel_86: two kinematics_final_velocity_from_distance calls — accel 5/dist 100 and accel 10/dist 200. "Both objects start from rest" → initial_velocity may be omitted or 0 (default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 409 OUT343

✓

FindVeganRestaurantsInThreeCitiesBFCL/parallel_91: three vegan_restaurant_find_nearby calls — LA, SF, Seattle, each with operating_hours 22 (prompt: "open until at least 22:00", a non-default value → required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 466 OUT300

✓

GenerateFourTravelItinerariesBFCL/parallel_90: four travel_itinerary_generator calls — Tokyo/7/200/urban, Paris/10/150/history, Sydney/5/100/nature, Rome/12/180/culture. Tokyo's "urban" equals the default → omit or "urban".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 817 OUT580

✓

GetDirectionsForRoundTripBFCL/parallel_89: three get_directions calls — SF->Palo Alto (fastest), Palo Alto->Golden Gate Bridge (scenic), Golden Gate Bridge->SF (fastest). route_type explicitly stated each time.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 516 OUT389

✓

MagneticFieldForTwoLoopsBFCL/parallel_95: two calculate_magnetic_field calls — current 10/radius 0.5 and current 15/radius 1.0. "Same as in free space" = default permeability → may be omitted or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 518 OUT359

✓

ResonantFrequencyForTwoCircuitsBFCL/parallel_97: two calculate_resonant_frequency calls — L 0.005/C 1e-07/round 3 and L 0.007/C 2e-07/round 4. Rounding is stated explicitly → required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 484 OUT328

✓

ShortestDrivingDistanceRoundTripBFCL/parallel_92: three get_shortest_driving_distance calls — NYC->LA, LA->Miami, Miami->NYC, all in miles (prompt explicitly "in miles" → non-default unit required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 474 OUT372

Tool Calling - Parallel Calls: Physical & Life Sciences (set 2) 13 passed

100.0%

IN111 763 OUT5 669

✓

AbsolutePressureAtTwoAltitudesBFCL/parallel_102: two calc_absolute_pressure calls — gauge 2.5 with atm 1 (default; may be omitted) and gauge 2.5 with atm 0.85 (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 523 OUT222

✓

BoilingMeltingPointsAcrossSeaLevelsBFCL/parallel_100: four get_boiling_melting_points calls — water@0, iron@1000, water@1000, iron@0. FLAG parallel_100: BORDERLINE. The prompt ("water and iron at sea levels of 0 and 1000 meters respectively") reads most naturally as TWO calls (water@0, iron@1000), but the BFCL ground truth is the full 2x2 cross product (FOUR calls). Kept BFCL-faithful to the ground truth; a competent human could reasonably answer with two calls, so this item's expected call count is not fully deterministic.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 409 OUT295

✓

CellInfoDetailedAndBriefBFCL/parallel_108: two biology_get_cell_info calls — neuron with detailed=true (prompt: "detailed description"), muscle with detailed=false ("less detailed", the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 412 OUT884

✓

DensityForTwoSubstancesBFCL/parallel_101: two calculate_density calls (mass/volume schema) — 10 kg / 2 m³ and 15 kg / 3 m³. unit "kg/m³" equals the default → may be omitted or "kg/m³".1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 518 OUT194

✓

ElectricFieldStrengthAtSeveralDistancesBFCL/parallel_98: four calculate_electric_field_strength calls — charge 2, distances 0.5/1/2 in vacuum (default medium), plus distance 1 in air. The distance-1 vacuum and distance-1 air calls are distinguished by medium. Vacuum equals the default → may be omitted or "vacuum"; "air" is required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 835 OUT408

✓

EntropyChangeForTwoConditionsBFCL/parallel_103: two entropy_change_calculate calls — substance A, mass 2, temps 25->75 and 10->50. pressure 1 equals the default → may be omitted or 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 646 OUT401

✓

FetchDnaSequenceThreeWaysBFCL/parallel_106: three fetch_DNA_sequence calls for id XYZ123 — default fasta/0, then genbank format, then 500 bp upstream. Defaults (fasta, upstream 0) may be omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 617 OUT327

✓

FunctionLookupForTwoMoleculesBFCL/parallel_110: two cell_biology_function_lookup calls — ATP in mitochondria and DNA in nucleus, both with specific_function=true (prompt asks for a specific function within the organelle).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 473 OUT360

✓

HeatCapacityForThreeConditionsBFCL/parallel_105: three calc_heat_capacity calls — 300K/2.5m³, 350K/2.5m³, 300K/1.5m³, all for air. "air" equals the default gas → may be omitted or "air".1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 694 OUT432

✓

MolecularWeightForTwoCompoundsBFCL/parallel_111: two calculate_molecular_weight calls — Glucose (C6H12O6) and Sucrose (C12H22O11), both in grams/mole (prompt explicitly requests that unit; ground truth also accepts "g/mol").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 474 OUT400

✓

PhaseChangeEnergyForWaterBFCL/parallel_99: four thermo_calculate_energy calls — 500 g water, phase melting/freezing/ vaporization/condensation. Substance "water" equals the default → may be omitted or "water".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 883 OUT701

✓

ProteinSequencesForTwoGenesTwoSpeciesBFCL/parallel_107: four get_protein_sequence calls — BRCA1 and BRCA2 in Homo sapiens (default species, may be omitted) and in Pan troglodytes (non-default → required).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 712 OUT445

✓

ProteinsInThreeCompartmentsBFCL/parallel_109: three cellbio_get_proteins calls — nucleus/mitochondria/cytoplasm, each with include_description=true (prompt asks for "a brief description of each protein").1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 567 OUT600

Tool Calling - Parallel Calls: Physical & Life Sciences (set 3) 13 passed

100.0%

IN113 625 OUT5 396

✓

AnalyzeDnaForThreeMutationTypesBFCL/parallel_114: six analyze_dna_sequence calls — two sequences ("AGCTTAGCTA" and "AGCTTAGGCTA") against reference "AGCTTAGCTA", each checked for insertion (default, may be omitted), deletion, and substitution mutations.1 AI query 6 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Passed

IN9 205 OUT749

✓

DiabetesPredictionForFourPeopleBFCL/parallel_113: four diabetes_prediction calls for four people with distinct weight/height/activity.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 966 OUT494

✓

EntropyChangeIsothermalAndNotBFCL/parallel_104: two calculate_entropy_change calls — same 300->350 K and heat_capacity 4.18, once isothermal (the default true, may be omitted) and once not isothermal (false, required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 628 OUT392

✓

ForestGrowthForTwoRegionsBFCL/parallel_120: two forest_growth_forecast calls — Amazon Rainforest over 10 years and Boreal Forests of Canada over 20 years, excluding human impact (false = the default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 519 OUT213

✓

GenerateFourDnaSequencesBFCL/parallel_123: four generate_DNA_sequence calls — length 500 with a single-nucleotide preference of A, T, C, G respectively.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 869 OUT432

✓

GeneticSimilarityInTwoFormatsBFCL/parallel_115: four genetics_calculate_similarity calls — human vs chimpanzee and human vs gorilla, each in percentage (default, may be omitted) and fraction. species1 accepts "human"/"Human".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 814 OUT400

✓

GenotypeFrequenciesForThreeGenotypesBFCL/parallel_116: three calculate_genotype_frequency calls — allele frequency 0.7, genotypes AA/Aa/aa.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 683 OUT381

✓

IdentifyFourBirdsBFCL/parallel_119: four identify_bird calls — small blue/forest (default size, may be omitted), large black/lake, medium brown/desert, large green/tropical rainforest.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 922 OUT518

✓

MutationTypeForTwoSnpsBFCL/parallel_112: two mutation_type_find calls — rs123456 in Homo sapiens (default species, may be omitted) and rs7891011 in Canis lupus familiaris (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 541 OUT294

✓

PopulationDensityForChinaTwoYearsBFCL/parallel_117: two calculate_density calls (country/population schema) — China in 2000 (pop 1.267 billion) and 2010 (pop 1.341 billion), land area 9.597 million km² both times.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 604 OUT309

✓

PrecipitationStatsForThreeTimeFramesBFCL/parallel_118: three ecology_data_precipitation_stats calls — Amazon rainforest for six_months, year, five_years.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 614 OUT319

✓

TurtlePopulationForTwoYearsBFCL/parallel_121: two ecology_get_turtle_population calls — Galapagos Islands in 2015 and 2020, both with species info requested (species=true; years stated explicitly → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 534 OUT483

✓

VehicleEmissionsForThreeVehiclesBFCL/parallel_122: three calculate_vehicle_emission calls — 15000 miles each: gas with default factor 1.4 (may be omitted), diesel with factor 2.7, EV with factor 0.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 726 OUT412

Tool Calling - Parallel Calls: Services & Lookups 12 passed

100.0%

IN98 440 OUT3 780

✓

CalculateShortestAndScenicDistancesForTwoLegsBFCL/parallel_190: four calculate_shortest_distance calls — {NYC->LA, LA->Miami} x {Shortest, Scenic}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 372 OUT532

✓

FetchNewsForTwoTopicsInEuropeBFCL/parallel_192: two get_news calls — Climate Change and Artificial Intelligence, 5 articles each, region Europe. Region "Europe" explicitly requested -> required (accept "Europe"/"EU").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 053 OUT211

✓

FindLibrariesWithFacilitiesInThreeCitiesBFCL/parallel_191: three public_library_find_nearby calls — NY[Reading Room,Fiction], LA[Wi-Fi,Children Section], Chicago[Cafe,Reading Room]. Facilities compared as an unordered pair.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 247 OUT333

✓

FindParksWithAmenitiesInThreeCitiesBFCL/parallel_189: three parks_find_nearby calls — NY[Tennis Court,Picnic Area], LA[Playground,Running Track], Chicago[Tennis Court,Playground]. Amenities compared as an unordered pair.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 252 OUT351

✓

FindUpcomingEventsInThreeCitiesBFCL/parallel_194: three event_finder_find_upcoming calls — jazz/LA(14d), rock/Chicago(10d), classical/Boston(7d). All day windows explicitly stated -> days_ahead required (7 for Boston, which also equals the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 265 OUT334

✓

GetCurrentWeatherForFourCitiesBFCL/parallel_199: four get_current_weather calls — New York, Los Angeles, London, Tokyo, with both temperature and humidity included. Prompt explicitly asks to include both -> flags == true (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 347 OUT324

✓

GetDetailedBriefsForTwoMoviesBFCL/parallel_195: two movie_details_brief calls — Inception and The Dark Knight, extra_info true. Prompt asks for Director/Cast/Awards -> extra_info == true required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 031 OUT415

✓

GetLawsuitDetailsForTwoCasesBFCL/parallel_196: two get_lawsuit_details calls — case 12345 (NY Supreme Court, with verdict) and case 67890 (LA Superior Court, no verdict). "would like the verdict" -> true; "do not need" -> null/false.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 180 OUT245

✓

GetLawsuitInfoForTwoCasesByYearBFCL/parallel_197: two lawsuit_info calls — 12345ABC (2018, New York) and 67890XYZ (2019, California). Years are explicitly stated -> required. First location "New York" may be omitted (matches the default).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 171 OUT257

✓

GetStockPricesForTwoCompaniesOverDaysBFCL/parallel_179: two get_stock_price (by-days variant) calls — Apple on NASDAQ for 10 days, Microsoft on NYSE for 15 days. Both exchanges are explicitly named -> required (NASDAQ / NYSE). The tool's 'company' field is a ticker symbol; company names are accepted as synonyms.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 054 OUT227

✓

SearchLawsuitsForTwoEntitiesInCaliforniaBFCL/parallel_198: two lawsuit_search calls — Google/Santa Clara and Facebook/San Mateo, state California. Prompt states "both in the state of California" -> state required (accept "California"/"CA").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 091 OUT240

✓

SendTwoEmailsWithCcAndBccBFCL/parallel_193: two send_email calls — Project Update to john.doe (cc manager, bcc hr) and Meeting Reminder to jane.doe (cc assistant, bcc hr). Bodies matched exactly.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 377 OUT311

Tool Calling - Parallel Calls: Statistics & Finance 13 passed

100.0%

IN117 665 OUT5 117

✓

CalculateBinomialProbabilitiesForBasketballShotsBFCL/parallel_163: three calc_binomial_prob calls — (10,6,0.6), (10,6,0.5), (15,6,0.5).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 161 OUT357

✓

CalculateCompoundInterestForThreePeriodsBFCL/parallel_170: three calculate_compound_interest calls — principal 5000, rate 0.025, time 2/3/5. Prompt states "compounded quarterly, 4 times in a year" -> n == 4 required (tightened; also ground truth).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 114 OUT428

✓

CalculateDiscountedCashFlowForTwoBondsBFCL/parallel_169: two calculate_discounted_cash_flow calls — (50,5,0.05) and (60,7,0.04). Prompt explicitly states "face value of the bond is $1000" for both -> face_value == 1000 required (ground truth also accepts omission via the default, but an explicitly-stated value must be supplied).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 998 OUT363

✓

CalculateEventProbabilitiesForCardDeckBFCL/parallel_162: three probability_of_event calls — Ace 4/52 as decimal, heart 13/52 as decimal, red card 26/52 as ratio. format_as_ratio: "as a decimal" -> false or omitted; "as a ratio" -> true (required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 146 OUT338

✓

CalculateQuarterlyDividendForThreeScenariosBFCL/parallel_168: three finance_calculate_quarterly_dividend_per_share calls — (5M,2M), (6M,2M), (6M,2.5M).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 184 OUT406

✓

CalculateReturnOnEquityForTwoCompaniesBFCL/parallel_171: two calculate_return_on_equity calls — (1M,5M,200k) and (2M,10M, no dividends). Second company "did not pay any dividends" -> dividends_paid null or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 959 OUT511

✓

CalculateRoiForThreeStocksBFCL/parallel_174: three calculate_return_on_investment calls — (150,180,20), (200,210,30), (250,300,40). Dividends are explicitly stated per stock -> required.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 100 OUT380

✓

ComputeRSquaredForTwoRegressionModelsBFCL/parallel_167: two linear_regression_get_r_squared calls on the same CSV — swap education/credit_score as dependent variable. Independent-variable order follows the prompt/ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 934 OUT294

✓

PredictFutureValueForTwoInvestmentsBFCL/parallel_172: two finance_predict_future_value calls — (5000,0.05,10) and (7000,0.04,15). Prompt states "compound interest annually" -> compounding_periods_per_year == 1 required (tightened; also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 944 OUT336

✓

PredictHousePricesForThreeHousesBFCL/parallel_166: three predict_house_price calls — NY 2500/3/2000, LA 3000/3/2005, Chicago 2000/2/1995. The second house is described as "similar" with only area/year/location changed, so rooms == 3 per ground truth.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 151 OUT547

✓

PredictProfitForTwoFundsBFCL/parallel_173: two investment_predictProfit calls — (5000,0.07,5) and (8000,0.05,7).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 920 OUT309

✓

RunChiSquaredTestForTwoContingencyTablesBFCL/parallel_164: two chi_squared_test calls — tables [45,55,35,65] and [30,70,50,50], alpha 0.05. Prompt states "significance level of 0.05 for both" -> alpha == 0.05 required (tightened). Table order is fixed.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 904 OUT335

✓

RunTTestAtTwoSignificanceLevelsBFCL/parallel_165: two t_test calls — same datasets, alpha 0.05 then 0.01 (both stated -> required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 150 OUT513

Tool Calling - Simple: Arts, Venues & Music (set 1) 8 passed

100.0%

IN50 817 OUT1 670

✓

SimpleConcertDetailsBeyonceBFCL/simple_python_286 (concert.get_details): artist=Beyonce, location=San Diego, date=April 2022.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 362 OUT92

✓

SimpleConcertInfoPopNyBFCL/simple_python_284 (concert_info.get): location=New York, date=next month, genre=Pop.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 337 OUT94

✓

SimpleGenerateMelodyC4BFCL/simple_python_291 (music_generator.generate_melody): key=C, start_note=C4, length=16, tempo=120 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 385 OUT175

✓

SimpleKeySignatureCsharpBFCL/simple_python_298 (music_theory.key_signature): key='C#', scale_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 322 OUT145

✓

SimpleMuseumHoursLouvreBFCL/simple_python_273 (museum.get_hours): name='Louvre Museum', location=Paris, day omitted or Monday.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 329 OUT137

✓

SimplePaintNeededWall30x12BFCL/simple_python_259 (calculate_paint_needed): coverage_rate=400, length=30, height=12.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 385 OUT138

✓

SimpleReligionHistoryChristianity16cBFCL/simple_python_252 (get_religion_history): religion=Christianity, century=16, sort_by=importance, count=10.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 362 OUT602

✓

SimpleSculptureInfoPlensaBFCL/simple_python_263 (get_sculpture_info): artist_name='James Plensa', detail=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 335 OUT287

Tool Calling - Simple: Arts, Venues & Music (set 2) 8 passed

100.0%

IN50 567 OUT1 495

✓

SimpleComposeMelodyCFGBFCL/simple_python_292 (compose_melody): progression=[C,F,G], measures=4, instrument=Piano or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 323 OUT111

✓

SimpleConcertFindDetailsWeekndBFCL/simple_python_290 (concert.find_details): artist=The Weeknd, month=December, year=2022 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 301 OUT110

✓

SimpleFindConcertRockChicagoBFCL/simple_python_285 (find_concert): location=Chicago, price=100, genre=Rock.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 311 OUT109

✓

SimpleMuseumInfoMetBFCL/simple_python_274 (museum_info): museum_name=Metropolitan Museum of Art, info_type=opening_hours or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 299 OUT149

✓

SimpleMusicalScaleCsharpBFCL/simple_python_299 (musical_scale): key='C#'/'C sharp', scale_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 290 OUT96

✓

SimplePaintRequirementWallExclusionBFCL/simple_python_260 (paint_requirement.calculate): area {width=20,height=12}, paint_coverage=350, exclusion {type=window, area=15}.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN6 409 OUT205

✓

SimpleReligionHistoryChristianity300To400BFCL/simple_python_254 (get_religion_history): religion=Christianity, start_year=300, end_year=400, event_type=all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 326 OUT532

✓

SimpleSculptureDetailsDavidBFCL/simple_python_264 (sculpture.get_details): artist=Michelangelo, title=David, detail=size.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 308 OUT183

Tool Calling - Simple: Arts, Venues & Music (set 3) 9 passed

100.0%

IN60 251 OUT1 597

✓

SimpleChordProgressionKeyBFCL/simple_python_297 (music.theory.chordProgression): progression=[I,V,vi,IV], bools any.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 704 OUT147

✓

SimpleCompoundInterest10kBFCL/simple_python_269 (calculate_compound_interest): principle=10000, interest_rate=0.05, time=10, compounds_per_year=1 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 740 OUT297

✓

SimpleConcertSearchClassicalLaBFCL/simple_python_287 (concert.search): genre=classical, location=Los Angeles/LA, date='this weekend', price_range=cheap.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 696 OUT108

✓

SimpleCreateMixCMajorBFCL/simple_python_293 (music_composer.create_mix): scale='C Major', note_duration=quarter, track_length=180.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 719 OUT97

✓

SimpleGenerateCircleImageRedBFCL/simple_python_256 (generate_circle_image): radius=50, color=Red, background=white or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 688 OUT91

✓

SimpleMuseumWorkingHoursLouvreBFCL/simple_python_276 (museum_working_hours.get): museum=Louvre, location=Paris, day=Monday or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 674 OUT95

✓

SimpleRetrieveReligionInfoBuddhismBFCL/simple_python_253 (retrieve_religion_info): religion_name=Buddhism, detail_level=full.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 669 OUT565

✓

SimpleSculptureSearchChicago19cBFCL/simple_python_265 (sculpture_search): location=Chicago, time_frame='19th century', material=all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 686 OUT85

✓

SimpleTimeDifferenceSfSydneyBFCL/simple_python_250 (get_time_difference): place1=San Francisco/SF, place2=Sydney.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 675 OUT112

Tool Calling - Simple: Arts, Venues & Music (set 4) 9 passed

100.0%

IN56 958 OUT1 521

✓

SimpleBuildingDimensionsEmpireStateBFCL/simple_python_270 (building.get_dimensions): building_name=Empire State Building, unit=feet.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 324 OUT128

✓

SimpleConcertFindNearbySeattleBFCL/simple_python_289 (concert.find_nearby): location=Seattle, genre=jazz.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 315 OUT96

✓

SimpleCreateChordProgressionCBFCL/simple_python_294 (music_generation.create_chord_progression): key=C, chords=4, progression_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 328 OUT94

✓

SimpleEarliestReferenceJesusBFCL/simple_python_251 (get_earliest_reference): name='Jesus Christ', source='historical records'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 322 OUT360

✓

SimpleFindInstrumentAcousticBFCL/simple_python_280 (find_instrument): budget=1000, type=acoustic, make omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 321 OUT88

✓

SimpleGuitarPriceLesPaulBFCL/simple_python_283 (guitar_price.find): model='Gibson Les Paul', condition=Excellent, location=Chicago.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 338 OUT104

✓

SimpleIdentifyColorSeaGreenBFCL/simple_python_257 (identify_color_rgb): color_name='Sea Green', standard=basic or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 319 OUT81

✓

SimpleMuseumInfoBritishDateBFCL/simple_python_277 (museum_info): museum='The British Museum', date=2023-06-20, information=[opening_hours,ticket_price] (either order).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN6 362 OUT290

✓

SimpleSculptureValueThinkerBFCL/simple_python_266 (get_sculpture_value): sculpture='The Thinker', artist=Rodin.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 329 OUT280

Tool Calling - Simple: Arts, Venues & Music (set 5) 8 passed1 failed

88.9%

IN61 089 OUT1 271

✓

SimpleCircleDimensionsRadius5BFCL/simple_python_272 (calculate_circle_dimensions): radius=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 388 OUT143

✓

SimpleDrawRectangleRedBFCL/simple_python_261 (draw_rectangle): width=20, height=10, color=red.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 416 OUT88

✓

SimpleFindFluteYamahaBFCL/simple_python_282 (find_flute): brand=Yamaha, specs=[open hole, C foot, silver headjoint].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 420 OUT122

✓

SimpleGenerateScaleProgressionCBFCL/simple_python_296 (music_generator.generate_scale_progression): key=C, tempo=80, duration=4, scale_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 423 OUT101

✓

SimpleInstrumentDetailsPianoYamahaBFCL/simple_python_278 (get_instrument_details): instrument=piano, manufacturer=Yamaha, features=[price,rating] (either order).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 404 OUT277

×

SimpleMetTopArtworksBFCL/simple_python_275 (metropolitan_museum.get_top_artworks): number=5, sort_by=popularity or omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 817 OUT233

✓

SimpleMixPaintYellowBlueBFCL/simple_python_258 (mix_paint_color): color1=yellow, color2=blue, lightness=60.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 410 OUT107

✓

SimplePapalBiographyInnocentIIIBFCL/simple_python_255 (religious_history.get_papal_biography): papal_name=Innocent III, include_contributions=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 399 OUT100

✓

SimpleSculptureLocatorMichelangeloBFCL/simple_python_268 (sculpture_locator.find_by_artist): artist=Michelangelo, material=Marble, location=Rome.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 412 OUT100

Tool Calling - Simple: Arts, Venues & Music (set 6) 6 passed1 failed

85.7%

IN41 913 OUT760

✓

SimpleAnalyzeStructureB1004BFCL/simple_python_271 (analyze_structure): building_id=B1004, floors=[2,3,4], mode=dynamic.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 010 OUT127

✓

SimpleConcertBookingEminemBFCL/simple_python_288 (concert_booking.book_ticket): artist=Eminem, city=New York City, num_tickets=2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 970 OUT97

✓

SimpleFindExhibitionNyBFCL/simple_python_267 (find_exhibition): location='New York City, NY', art_form=sculpture, month omitted, user_ratings=high.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 986 OUT114

✓

SimpleInstrumentInfoViolin1721BFCL/simple_python_281 (get_instrument_info): name=Violin, maker=Stradivarius, year=1721.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 004 OUT107

×

SimpleInstrumentPriceStratocasterBFCL/simple_python_279 (instrument_price.get): brand=Fender, model='American Professional II Stratocaster', finish=Rosewood.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.finish?.ToLower()).AnyOf(["rosewood"])

Failed

IN5 985 OUT107

✓

SimpleModifyPaintingOilBFCL/simple_python_262 (modify_painting): size='12x18', medium=oil, dominant_color=red.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 987 OUT115

✓

SimpleSongLyricsBohemianBFCL/simple_python_295 (get_song_lyrics): song_title='Bohemian Rhapsody', artist_name=Queen, lang=English or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 971 OUT93

Tool Calling - Simple: Food, Travel & Shopping (set 1) 8 passed

100.0%

IN51 800 OUT1 084

✓

SimpleCookingConvertPoundsToOuncesBFCL/simple_python_365 (cooking_conversion.convert): 2 pounds->ounces of butter.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 470 OUT88

✓

SimpleCurrencyExchangeUsdToCadBFCL/simple_python_388 (currency_exchange.convert): USD->CAD, amount=500.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 470 OUT96

✓

SimpleGroceryStoreOrganicHoustonBFCL/simple_python_369 (grocery_store.find_nearby): Houston, categories {Organic,Fruits,Vegetables} any order.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 467 OUT145

✓

SimpleHotelBookPetFriendlyManhattanBFCL/simple_python_380 (hotel_booking): Manhattan, single, 3 nights, 2023-03-10, pet_friendly.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN6 533 OUT145

✓

SimpleMapsDistanceEiffelLouvreBFCL/simple_python_394 (maps.get_distance_duration): Eiffel Tower -> Louvre Museum, traffic omitted/false.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 471 OUT103

✓

SimpleRecipeFindByDietMealBFCL/simple_python_353 (find_recipes): diet=gluten-free, meal_type=dinner, ingredients omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 451 OUT317

✓

SimpleRestaurantFinderItalianNYCBFCL/simple_python_361 (restaurant_finder): city=NYC, cuisine=Italian, diet=Gluten-free.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 466 OUT103

✓

SimpleTimeZoneLondon24hBFCL/simple_python_376 (time_zone_converter): city=London, country=UK, display_format=24h.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 472 OUT87

Tool Calling - Simple: Food, Travel & Shopping (set 2) 7 passed2 failed

77.8%

IN67 314 OUT1 656

✓

SimpleBestSushiPlacesTokyoBFCL/simple_python_362 (get_best_sushi_places): city=Tokyo, top=5, review_rate=4.0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 058 OUT241

✓

SimpleCurrencyConverterUsdToGbpBFCL/simple_python_389 (currency_converter): USD->GBP, amount=200.0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 053 OUT102

✓

SimpleCurrentTimeSydneyCityBFCL/simple_python_377 (get_current_time): city=Sydney, country=Australia, format omitted/HH:MM:SS.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 024 OUT172

✓

SimpleDistanceCalcNyToBostonTerrainBFCL/simple_python_397 (distance_calculator.calculate): New York -> Boston, consider_terrain=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 040 OUT94

✓

SimpleHiltonAvailabilityParisBFCL/simple_python_381 (hilton_hotel.check_availability): Paris, 2023-04-04..08, 2 adults, chain Hilton/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 104 OUT143

×

SimpleRecipeCaloriesBeefLasagnaBFCL/simple_python_355 (recipe_info.get_calories): website=Foodnetwork.com, recipe=Beef Lasagna, meal_time omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 894 OUT397

✓

SimpleRecipeUnitTbspToTspBFCL/simple_python_366 (recipe.unit_conversion): 2 tablespoon->teaspoon, precision omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 044 OUT77

×

SimpleSafewayOrderPaloAltoBFCL/simple_python_370 (safeway.order): Palo Alto, [olive oil, rice], [3,1]. EXPECTED MODEL MISS (kept, do not loosen): "a five pound bag of rice" = quantity 1 (5 is the weight, not the count); models tend to send quantity 5. This is a genuine model error the test correctly catches.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.quantity[1] == 1 Values: args.quantity[1] = 5

Failed

IN7 060 OUT108

✓

SimpleVeganSoupRecipeUnder30BFCL/simple_python_354 (get_vegan_recipe): dish_type=soup, cooking_time=30, ingredient_preference omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 037 OUT322

Tool Calling - Simple: Food, Travel & Shopping (set 3) 8 passed

100.0%

IN51 987 OUT818

✓

SimpleBookHotelHiltonChicagoBFCL/simple_python_382 (book_hotel): Hilton Hotel, Chicago, single, 2022-12-10, 2 nights.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN6 547 OUT133

✓

SimpleCookingTimeRoastChickenBFCL/simple_python_368 (calculate_cooking_time): weight_kg=1.5, method omitted/roast, temp omitted/180.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 488 OUT143

✓

SimpleCurrencyConvertEurToCadBFCL/simple_python_390 (currency_conversion.convert): 150 EUR->CAD.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 487 OUT91

✓

SimpleCurrentTimeSydneyLocationBFCL/simple_python_379 (get_current_time): location=Sydney, country=Australia, timezone omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 476 OUT67

✓

SimpleParkingLotCentralParkBFCL/simple_python_395 (parking_lot.find_nearest): Central Park NY, radius=2, type omitted/public.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 493 OUT101

✓

SimpleRecipeFinderVeganServes2BFCL/simple_python_356 (recipe_finder.find): servings=2, diet=vegan, prep_time=30.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 512 OUT98

✓

SimpleRestaurantClosestSushiBostonBFCL/simple_python_363 (restaurant_search.find_closest): Boston, Sushi, amenities=[Patio].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 491 OUT94

✓

SimpleWholeFoodsPriceLABFCL/simple_python_371 (whole_foods.check_price): Los Angeles, [tomatoes, lettuce].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 493 OUT91

Tool Calling - Simple: Food, Travel & Shopping (set 4) 7 passed

100.0%

IN42 018 OUT1 093

✓

SimpleBookRoomThePlazaBFCL/simple_python_383 (book_room): The Plaza, single, 2 nights.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 004 OUT94

✓

SimpleConvertCurrencyJpyToUsdBFCL/simple_python_393 (convert_currency): 20000 JPY->USD.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 010 OUT134

✓

SimpleFindRestaurantItalianBrooklynBFCL/simple_python_364 (find_restaurant): Brooklyn, Italian, Gluten-free.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 999 OUT94

✓

SimpleGetRecipeVeganChocCakeBFCL/simple_python_357 (get_recipe): dish_name=(vegan) chocolate cake, diet_preference=vegan.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 991 OUT347

✓

SimpleHospitalLocateDenverBFCL/simple_python_396 (hospital.locate): Denver CO, radius=5, department=Pediatrics.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 006 OUT160

✓

SimpleTimezoneConvertNyToLondonBFCL/simple_python_378 (timezone.convert): 3pm, New York -> London.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 002 OUT109

✓

SimpleWholeFoodsTopBrandsBananasBFCL/simple_python_372 (whole_foods.find_top_brands): product=bananas, number omitted/5, organic=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 006 OUT155

Tool Calling - Simple: Food, Travel & Shopping (set 5) 5 passed1 failed

83.3%

IN38 665 OUT1 116

✓

SimpleExchangeRateFeeGbpJpyBFCL/simple_python_391 (get_exchange_rate_with_fee): GBP->JPY, fee=0.02.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 898 OUT98

✓

SimpleHotelBookingBookParisBFCL/simple_python_384 (hotel_booking.book): Paris, 07-10-2022..07-20-2022, 2 adults, 1 child, Standard/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 961 OUT148

✓

SimpleMuseumHoursMetSaturdayBFCL/simple_python_398 (get_museum_hours): museum=Metropolitan Museum of Art/The Met, day=Saturday.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 877 OUT90

✓

SimpleRecipeSearchGfCookieBFCL/simple_python_358 (recipe_search): diet=[gluten free], time_limit=30, dish=cookie.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 888 OUT336

×

SimpleRestaurantSearchRatedNYCBFCL/simple_python_399 (restaurant_search): NYC, Italian, rating=4, accepts_credit_cards=true.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 089 OUT282

✓

SimpleWalmartPurchaseSanJoseBFCL/simple_python_373 (walmart.purchase): San Jose, [apples, rice, (bottled) water], pack_size=[1,1,12].1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 952 OUT162

Tool Calling - Simple: Food, Travel & Shopping (set 6) 5 passed

100.0%

IN27 831 OUT819

✓

SimpleGameHighestScoringPlayerBFCL/simple_python_350 (get_highest_scoring_player): game=Valorant, season=2022.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 558 OUT81

✓

SimpleHotelBookRoomKingLABFCL/simple_python_385 (hotel_bookings.book_room): LA, king size, 15-10-2023, 2 nights, rooms omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 605 OUT173

✓

SimpleLatestExchangeGbpJpyBFCL/simple_python_392 (latest_exchange_rate): GBP->JPY, amount omitted/1.0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 544 OUT92

✓

SimpleNutritionalInfoAvocadoBFCL/simple_python_374 (grocery_info.nutritional_info): Walmart, avocado, [Protein,Calories,Carbohydrates].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 565 OUT163

✓

SimpleRecipeSearchVegPastaBFCL/simple_python_359 (recipe_search): dietary_restriction=Vegetarian, ingredients=[pasta,cheese], servings=2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 559 OUT310

Tool Calling - Simple: Food, Travel & Shopping (set 7) 4 passed

100.0%

IN20 861 OUT449

✓

SimpleBookHotelLuxuryParadiseBFCL/simple_python_386 (book_hotel): Hotel Paradise, Las Vegas, luxury, 05-12-2022, 3 days, view=city.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 267 OUT151

✓

SimpleFindRecipeCaloriesCarbonaraBFCL/simple_python_360 (find_recipe): recipeName=pasta carbonara, maxCalories=500.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 197 OUT82

✓

SimpleGameMultiplayerFinderBFCL/simple_python_351 (multiplayer_game_finder): platform=Windows 10, rating=4.5, genre omitted/Action.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 200 OUT117

✓

SimpleWalmartPriceCheckPumpkinsBFCL/simple_python_375 (walmart.check_price): [pumpkins,eggs], quantities [3,24] or [3,2], store omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 197 OUT99

Tool Calling - Simple: Food, Travel & Shopping (set 8) 3 passed

100.0%

IN14 610 OUT604

✓

SimpleFindRecipeVeganBrowniesBFCL/simple_python_367 (find_recipe): dietary_restrictions=vegan, recipe_type=dessert (brownies), time=30.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 852 OUT360

✓

SimpleGameAverageUserScoreBFCL/simple_python_352 (gamespot.getAverageUserScore): game=Zelda BotW, platform=Nintendo Switch/all platforms.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 862 OUT94

✓

SimpleHotelBookingPlazaRangeBFCL/simple_python_387 (hotel_booking): Plaza Hotel, New York NY, 2022-06-01..2022-06-04, rooms omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 896 OUT150

Tool Calling - Simple: Knowledge, Places & History (set 1) 9 passed1 failed

90.0%

IN73 001 OUT1 871

✓

SimpleAirQualityChicagoBFCL/simple_python_203 (get_air_quality): location=Chicago, detail=true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 927 OUT112

✓

SimpleEvolutionScientistBFCL/simple_python_242 (get_scientist_for_discovery): discovery=Theory of Evolution.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 918 OUT124

✓

SimpleGermanWarEventsBFCL/simple_python_231 (history.get_key_events): country=Germany/DE, 1871-1945, event_type=[War].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 961 OUT524

✓

SimpleKingOfFranceBFCL/simple_python_230 (historic_leader_search): location=France, date=1510, title omitted or King.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 932 OUT98

✓

SimplePandaPopulationBFCL/simple_python_201 (estimate_population): species=panda(s), country=China/CN, year omitted or 2024.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 919 OUT119

✓

SimplePersonalityTraitsTypeBFCL/simple_python_227 (get_personality_traits by type): type=ENFJ, traits=[strengths, weaknesses].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 930 OUT343

✓

SimplePresidentCivilWarBFCL/simple_python_238 (us_history.get_president): event=American Civil War, year in 1861-1865.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 929 OUT146

×

SimpleRestaurantNearbyBFCL/simple_python_204 (restaurant.find_nearby): location=Seattle[, WA], cuisine=Chinese, max_distance=10.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 609 OUT230

✓

SimpleTordesillasDateBFCL/simple_python_233 (european_history.get_event_date): event=Treaty of Tordesillas, format=YYYY.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 943 OUT79

✓

SimpleTrafficBostonNewYorkBFCL/simple_python_205 (get_traffic_info): start=Boston, end=New York/NYC, mode omitted or driving.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 933 OUT96

Tool Calling - Simple: Knowledge, Places & History (set 2) 9 passed1 failed

90.0%

IN68 984 OUT1 437

✓

SimpleBitcoinNewsBFCL/simple_python_210 (get_news): topic=Bitcoin, quantity=5, region omitted or US.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 894 OUT163

×

SimpleEuropeanWars19cBFCL/simple_python_234 (history_eu.fetch_events): century=19, region=any of N/S/E/W, category=Wars. EXPECTED MODEL MISS (kept, do not loosen): region is an enum (Northern/Southern/Eastern/Western); the prompt says only "European", so any of the four is accepted, but models often send the invalid value "Europe". Failing to respect the enum is a genuine model error the test correctly catches.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.region.ToLower().AnyOf(["northern", "southern", "eastern", "western"])

Failed

IN6 909 OUT312

✓

SimpleLibraryNearbyBFCL/simple_python_209 (public_library.find_nearby): location=Boston MA/Massachusetts, facilities=[Fiction, Wi-Fi] (any order).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 919 OUT88

✓

SimpleMonarchOfEnglandBFCL/simple_python_232 (monarch.getMonarchOfYear): location=England/ENG, year=1800, fullName=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 911 OUT142

✓

SimpleNeutronDiscovererBFCL/simple_python_243 (get_discoverer): discovery=neutron, detail=true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 890 OUT105

✓

SimpleParksNearbyBFCL/simple_python_206 (parks.find_nearby): location=London[, UK], amenities=[Tennis Court].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 896 OUT179

✓

SimplePersonalityTraitsHobbyBFCL/simple_python_228 (get_personality_traits by hobby): hobby=jogging, trait_count=3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 893 OUT185

✓

SimplePopulationGrowthBFCL/simple_python_221 (population_growth_estimate): location=London, years=5, rate omitted or 1.2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 896 OUT86

✓

SimplePresidentDuringEventBFCL/simple_python_241 (US_President_During_Event): event=Civil War, country omitted or USA.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 889 OUT107

✓

SimpleTreatyLisbonDateBFCL/simple_python_235 (get_event_date): event=Treaty of Lisbon (variants), location omitted or Lisbon[, Portugal].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 887 OUT70

Tool Calling - Simple: Knowledge, Places & History (set 3) 10 passed

100.0%

IN72 377 OUT1 501

✓

SimpleAppleStockInfoBFCL/simple_python_212 (get_stock_info): company=Apple Inc., detail_level=detailed, market omitted or NASDAQ.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 223 OUT428

✓

SimpleCivilWarStartDateBFCL/simple_python_236 (us_history.get_event_info): event=American Civil War/Civil War, specific_info=Start Date.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 224 OUT78

✓

SimpleFlightBookBFCL/simple_python_213 (flight.book): departure=San Francisco/SF, destination=London, date=2022-04-27 (variants), time=afternoon, direct=true.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 278 OUT128

✓

SimpleGravitationPublicationYearBFCL/simple_python_244 (publication_year.find): author=Isaac Newton, work=Law of Universal Gravitation (variants), location omitted or all.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 234 OUT95

✓

SimpleMovieBriefBFCL/simple_python_215 (movie_details.brief): title=Interstellar, extra_info omitted or false.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 213 OUT213

✓

SimplePresidentIn1861BFCL/simple_python_239 (US_president.in_year): year=1861, full_name omitted or true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 241 OUT95

✓

SimpleRadiumDiscovererBFCL/simple_python_245 (discoverer.get): element_name=radium (quote variants), year omitted or 0, first omitted or true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 203 OUT69

✓

SimpleSendEmailBFCL/simple_python_211 (send_email): to=john.doe@example.com, subject=Meeting, body (opt trailing period), cc/bcc empty.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 280 OUT108

✓

SimpleShortestDistanceBFCL/simple_python_207 (calculate_shortest_distance): start=New York (variants), end=Miami (variants), route_preference=Shortest.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 247 OUT90

✓

SimpleUpcomingConcertsBFCL/simple_python_214 (event_finder.find_upcoming): location=New York (variants), genre=Rock, days_ahead=30.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 234 OUT197

Tool Calling - Simple: Knowledge, Places & History (set 4) 10 passed

100.0%

IN71 846 OUT1 307

✓

SimpleBmiCalcBFCL/simple_python_222 (calculate_bmi): weight=70, height=180, unit omitted or metric.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 183 OUT84

✓

SimpleCarEmissionsBFCL/simple_python_200 (calculate_emissions): distance=12000, fuel_type=gas, fuel_efficiency omitted or 25.0, efficiency_reduction omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 217 OUT260

✓

SimpleFmriAnalyzeBFCL/simple_python_217 (fMRI.analyze): data_source path, sequence_type=multi-band, smooth=6, voxel_size=2.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 226 OUT137

✓

SimpleGetDirectionsBFCL/simple_python_208 (map_service.get_directions): start=New York/NYC, end=Los Angeles/LA, avoid=[highways, tolls] (any order).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 177 OUT107

✓

SimpleGravityDiscoveryBFCL/simple_python_246 (science_history.get_discovery_details): discovery=Gravity, method_used omitted or default.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 143 OUT144

✓

SimpleGroupDynamicsBFCL/simple_python_223 (group_dynamics.pattern): total=50, extroverts=15, introverts=35.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 203 OUT190

✓

SimpleMostFollowedBFCL/simple_python_224 (social_media_analytics.most_followed): topic=psychology, sub_topics=[behaviour, group dynamics], region omitted or all.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 182 OUT107

✓

SimpleNeuronCoordinatesBFCL/simple_python_219 (get_neuron_coordinates): neuron_type=GABA, brain_region=All/all part of the brain/entire brain.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 178 OUT97

✓

SimplePresidentIn1940BFCL/simple_python_240 (history_api.get_president_by_year): year=1940, full_term_only omitted/true/false.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 155 OUT90

✓

SimpleSentimentAnalysisBFCL/simple_python_216 (sentiment_analysis): text=the review, language=english/en.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 182 OUT91

Tool Calling - Simple: Knowledge, Places & History (set 5) 9 passed1 failed

90.0%

IN73 127 OUT1 637

✓

SimpleBigFiveScoresBFCL/simple_python_229 (get_bigfive_scores): characteristics=[efficient, organized, easy going, compassionate], scale omitted or medium.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 320 OUT196

✓

SimpleChristianityHistoryBFCL/simple_python_249 (religion.history_info): religion=Christianity, till_century=14, include_people omitted or false.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 297 OUT372

✓

SimpleEinsteinContributionBFCL/simple_python_247 (historical_contrib.get_contrib): scientist=Albert Einstein, date=1915-03-17 (variants), category omitted or all.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 314 OUT162

✓

SimpleHistoricalGdpBFCL/simple_python_237 (get_historical_GDP): country=United States/US, start_year=1960, end_year=2000.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 319 OUT131

✓

SimpleMriReportBFCL/simple_python_218 (patient.get_mri_report): patient_id=546382, mri_type omitted or brain, status=concluded.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 328 OUT99

✓

SimpleNeuronalActivityBFCL/simple_python_220 (calculate_neuronal_activity): input_synaptic_rate=200, weight=0.5, decay_rate=0.1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 338 OUT125

✓

SimpleReadingPreferenceBFCL/simple_python_225 (psych_research.get_preference): category=reading, option_one=digital[reading], option_two=physical book(s), demographic omitted or all.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 301 OUT273

✓

SimpleRelativityInventionBFCL/simple_python_248 (science_history.get_invention): invention_name=theory of relativity, want_year=true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 291 OUT92

×

SimpleRenewableEmissionSavingsBFCL/simple_python_202 (calculate_emission_savings): energy_type=renewable, usage_duration=3, region=California/CA. EXPECTED MODEL MISS (kept, do not loosen): the prompt says "renewable energy sources" generically; models often over-specify a concrete type (e.g. "solar"). Ground truth wants "renewable".1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.energy_type.ToLower() == "renewable"

Failed

IN7 320 OUT105

✓

SimpleZodiacCompatibilityBFCL/simple_python_226 (get_zodiac_compatibility): sign1=Aries, sign2=Gemini, scale omitted or percentage.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 299 OUT82

Tool Calling - Simple: Law, Finance & Records (set 1) 10 passed

100.0%

IN73 051 OUT1 398

✓

SimpleAirQualityByDateBFCL/simple_python_196 (air_quality): location London, date reformatted to 08-16-2022.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 291 OUT98

✓

SimpleCivilCaseDetailsWithDissentBFCL/simple_python_167 (law.civil.get_case_details): case_title Roe v. Wade, include_dissent true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 291 OUT93

✓

SimpleCompoundedInterestQuarterlyBFCL/simple_python_153 (calculate_compounded_interest): principal 5000, rate 0.03, time 5, n 4 (quarterly).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 327 OUT350

✓

SimpleCrimeRateByCityBFCL/simple_python_164 (get_crime_rate): city San Francisco, state California/CA, type violent/omitted, year 2020.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 301 OUT110

✓

SimpleCriminalRecordsByPersonBFCL/simple_python_158 (get_criminal_records): name Mr. X, location New York, NY, from_year 2012, to_year 2015.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 324 OUT123

✓

SimpleDetailedWeatherForecastHoursBFCL/simple_python_185 (detailed_weather_forecast): location New York, duration 72 hours, include_precipitation true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 294 OUT89

✓

SimpleFutureValueByDurationBFCL/simple_python_155 (calculate_future_value): initial_investment 1000, interest_rate 0.05, duration 2, compounded omitted or 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 313 OUT169

✓

SimpleLawsuitDetailsWithExtraFieldsBFCL/simple_python_178 (get_lawsuit_details): case_number LAX2019080202, court_location Los Angeles, additional_details omitted or full default list.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 311 OUT113

✓

SimpleLawsuitSearchByFilingDateBFCL/simple_python_168 (lawsuit_search): company Google/GOOG, start_date 01-01-2021, location California, status ongoing/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 320 OUT122

✓

SimpleLawyerExperienceMonthsBFCL/simple_python_175 (lawyer.get_experience): name John Doe, law_type Bankruptcy.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 279 OUT131

Tool Calling - Simple: Law, Finance & Records (set 2) 10 passed

100.0%

IN70 861 OUT1 136

✓

SimpleAirQualityIndexByTimeBFCL/simple_python_197 (get_air_quality_index): location San Diego, time 12pm/12:00.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 059 OUT74

✓

SimpleCivilCasesRetrieveTheftBFCL/simple_python_165 (civil_cases.retrieve): year 2020, crime_type theft, location Los Angeles.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 074 OUT100

✓

SimpleFindNearbyDivorceLawyerBFCL/simple_python_166 (lawyer.find_nearby): city Chicago IL, specialty [Divorce], fee 400. NOTE: single-call contract kept intentionally; the model occasionally emits a spurious second call (no other visible tool fits this query), which is a model-reliability signal, not a test defect.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 086 OUT120

✓

SimpleGenerateLawContractRentalBFCL/simple_python_162 (generate_law_contract): parties [John, Alice], contract_type Rental Agreement, location California/CA.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 073 OUT97

✓

SimpleLawsuitDetailsFindByCompanyBFCL/simple_python_176 (lawsuit_details.find): company_name Apple Inc., year 2010, case_type Patent/IPR.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 085 OUT104

✓

SimpleLawsuitDetailsWithVerdictBFCL/simple_python_181 (get_lawsuit_details): case_number 123456-ABC, court_location Los Angeles, with_verdict true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 097 OUT118

✓

SimpleLocalNurseryAnnualPlantsBFCL/simple_python_193 (local_nursery.find): location Toronto, plant_types [Annual].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 063 OUT91

✓

SimplePropertyRecordsWithOwnerBFCL/simple_python_163 (property_records.get): address 123 main street, parcel_number 1234567890, county Santa Clara, include_owner true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 145 OUT142

✓

SimpleRoiSavingsAccountBFCL/simple_python_150 (calculate_roi): deposit 1000, annual_interest_rate 0.03, years 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 107 OUT162

✓

SimpleWeatherForecastDetailedDaysBFCL/simple_python_189 (weather_forecast_detailed): location New York, days 3, details true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 072 OUT128

Tool Calling - Simple: Law, Finance & Records (set 3) 9 passed1 failed

90.0%

IN71 842 OUT1 958

✓

SimpleCaseInfoVictimBFCL/simple_python_160 (get_case_info): docket 2022/AL2562, court California/CA, info_type victim.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 206 OUT106

✓

SimpleCrimeStatuteTheftBFCL/simple_python_161 (crime_statute_lookup): jurisdiction California/CA, crime theft, detail_level detailed.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 182 OUT452

✓

SimpleCurrentWeatherCelsiusBFCL/simple_python_186 (current_weather_condition): city Tokyo, country Japan, measurement c/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 182 OUT131

✓

SimpleEnvironmentalAirQualityDaysBFCL/simple_python_199 (environmental_data.air_quality_index): location San Jose, days 3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 166 OUT95

✓

SimpleFelonyCrimeRecordBFCL/simple_python_156 (crime_record.get_record): case_number CA123456, county San Diego, details true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 199 OUT110

✓

SimpleHighestGrossingBankBFCL/simple_python_151 (highest_grossing_banks): country U.S variants, year 2020, top_n 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 190 OUT241

✓

SimpleHistoricalFraudCaseBFCL/simple_python_170 (law_case_search.find_historical): subject fraud, from_year 2010, to_year 2015.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 198 OUT104

×

SimpleLawsuitCasesByCompanyYearBFCL/simple_python_177 (get_lawsuit_cases): company_name Facebook, year 2018, status all/omitted. EXPECTED MODEL MISS (kept, do not loosen): "Patent" is the case TYPE, not the status (enum is open/closed/all); status should be omitted/"all". Models often wrongly set status="Patent".1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.status?.ToLower()).AnyOf([null, "", "all"])

Failed

IN7 182 OUT316

✓

SimpleLawsuitInfoByCaseNumberBFCL/simple_python_182 (lawsuit_info): case_number XYZ123, year omitted/2023, location omitted/all.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 161 OUT76

✓

SimplePlantsForSlopeErosionBFCL/simple_python_194 (get_plants_for_slope): slope_type hill/steep/moderate, num_results 3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 176 OUT327

Tool Calling - Simple: Law, Finance & Records (set 4) 10 passed

100.0%

IN73 586 OUT1 640

✓

SimpleActDetailsAmendmentBFCL/simple_python_159 (get_act_details): act_name Criminal Law Amendment Act, amendment_year 2013.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 332 OUT132

✓

SimpleCheckFeloniesByNameBFCL/simple_python_157 (criminal_history.check_felonies): full_name John Doe, birth_date 01-01-1980, state California/CA.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 372 OUT109

✓

SimpleCourtCaseSearchNoFullTextBFCL/simple_python_169 (court_case.search): docket_number 123456, location Texas, full_text false/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 359 OUT101

✓

SimpleCurrentWeatherTempHumidityBFCL/simple_python_187 (get_current_weather): location Seattle, include_temperature true/omitted, include_humidity true/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 325 OUT127

✓

SimpleLawsuitCheckClosedStatusBFCL/simple_python_184 (lawsuit.check_case): case_id 1234, closed_status true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 345 OUT81

✓

SimpleLawsuitsSearchByCompanyLocationBFCL/simple_python_180 (lawsuits_search): company_name Google, location California/CA, year 2020, case_type all/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 340 OUT99

✓

SimpleMutualFundBalanceBFCL/simple_python_152 (calculate_mutual_fund_balance): investment_amount 50000, annual_yield 0.05, years 3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 370 OUT205

✓

SimpleSlopeGradientCoordinatesBFCL/simple_python_192 (calculate_slope_gradient): point1 [40.7128,-74.006], point2 [34.0522,-118.2437], unit degree/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 427 OUT211

✓

SimpleStateLawCaseYearRangeBFCL/simple_python_173 (law_case_search): topic land disputes, year_range [2015,2021], location New York, judicial_system state.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 381 OUT128

✓

SimpleTopConstitutionalCasesBFCL/simple_python_174 (get_top_cases): field_of_law constitutional law, top_number 10, country China/CN.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 335 OUT447

Tool Calling - Simple: Law, Finance & Records (set 5) 10 passed

100.0%

IN72 211 OUT1 760

✓

SimpleCarbonFootprintLifestyleBFCL/simple_python_195 (calculate_carbon_footprint): daily_miles 20, meat_meals_per_week 3, annual_trash_weight 500, flights_per_year omitted/0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 267 OUT138

✓

SimpleDailyWaterIntakeWeightBFCL/simple_python_198 (calculate_daily_water_intake): weight 70, activity_level omitted/moderate, climate omitted/temperate.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 193 OUT110

✓

SimpleFetchLawCaseByNumberBFCL/simple_python_171 (fetch_law_case_details): case_number 43403, court New York, year 2018.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 237 OUT113

✓

SimpleFutureValueByPresentValueBFCL/simple_python_154 (calculate_future_value): present_value 5000, annual_interest_rate 0.05, years 10, compounds_per_year omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 246 OUT172

✓

SimpleHumidityForecastDaysBFCL/simple_python_188 (weather.humidity_forecast): location Miami, days 7, min_humidity omitted/0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 202 OUT176

✓

SimpleLatestCourtCaseCompaniesBFCL/simple_python_179 (find_latest_court_case): company1 Apple, company2 Samsung, country USA/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 205 OUT93

✓

SimpleLawsuitSearchByCountyBFCL/simple_python_183 (lawsuit_search): entity Apple, county Santa Clara County, state California/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 193 OUT86

✓

SimpleLegalCaseFetchDetailsBFCL/simple_python_172 (legal_case.fetch): case_id R vs Adams, details true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 259 OUT143

✓

SimpleParkElevationAreaBFCL/simple_python_190 (park_information): park_name Yellowstone, information [Elevation, Area] (order-independent).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 196 OUT347

✓

SimpleTallestMountainsRadiusBFCL/simple_python_191 (locate_tallest_mountains): location Denver, radius 50, amount 5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 213 OUT382

Tool Calling - Simple: Live Answer Tool (isolated) (set 1) 1 passed

100.0%

IN4 013 OUT86

✓

SimpleAnswerStringLogistic1BFCL/live_simple_32-9-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L113: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 013 OUT86

Tool Calling - Simple: Live Answer Tool (isolated) (set 2) 1 passed

100.0%

IN3 980 OUT74

✓

SimpleAnswerStringLogistic2BFCL/live_simple_33-10-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L114: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 980 OUT74

Tool Calling - Simple: Live Answer Tool (isolated) (set 3) 1 skipped

-

IN0 OUT0

–

SimpleAnswerStringValidBFCL/live_simple_34-11-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L115: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Harness-semantic mismatch (item flaw for our runner): the query ('...Valid?') is directly answerable, and our system prompt permits answering in plain text when no tool is needed, so the model reasonably returns 0 tool calls instead of routing to the answer_string meta-tool. Not a deterministic tool-routing test. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Simple: Live Answer Tool (isolated) (set 4) 1 failed

0.0%

IN1 993 OUT129

×

SimpleAnswerStringWhatIsBFCL/live_simple_35-12-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L116: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 0

Failed

IN1 993 OUT129

Tool Calling - Simple: Live Assistants, Weather & Devices (set 1) 2 passed1 failed

66.7%

IN14 749 OUT322

✓

SimpleChangeFoodBurgerBFCL/live_simple_20-4-0 (change_food): food_item='burger', modification_request='no onions, extra cheese'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 903 OUT85

×

SimpleThinqAirCleanHighBFCL/live_simple_40-17-0 (ThinQ_Connect): AIR_CLEAN mode, HIGH wind, monitoring on, air clean POWER_ON. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. windStrength broadened (high|strongest|strong) for "strongest wind strength". EXPECTED MODEL MISS: 27B sets airConJobMode=AIR_CLEAN but omits airCleanOperationMode=POWER_ON — the prompt asks to "start the air purifying function", so POWER_ON is the fair GT; kept strict (a real tool-use miss).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.airCleanOperationMode?.ToLower()).AnyOf(["power_on"])

Failed

IN4 943 OUT105

✓

SimpleWeatherTelAvivBFCL/live_simple_4-3-0 (get_current_weather): location='Tel Aviv, Israel', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 903 OUT132

Tool Calling - Simple: Live Assistants, Weather & Devices (set 10) 3 passed

100.0%

IN13 964 OUT309

✓

SimpleChafodOrderPizzaBFCL/live_simple_23-5-1 (ChaFod): TheFod='PIZZA'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 630 OUT84

✓

SimpleUberEatMcdonaldsPizzaBFCL/live_simple_29-7-2 (uber.eat.order): restaurant="McDonald's", items=[pizza], quantities=[1].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 664 OUT93

✓

SimpleWeatherShanghaiBFCL/live_simple_13-3-9 (get_current_weather): location='Shanghai, China', unit=celsius. Prompt is Chinese, copied verbatim.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 670 OUT132

Tool Calling - Simple: Live Assistants, Weather & Devices (set 11) 3 passed

100.0%

IN15 112 OUT332

✓

SimpleChafodSwitchBurgerBFCL/live_simple_24-5-2 (ChaFod): TheFod='BURGER'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 009 OUT61

✓

SimpleFindBeerSierraNevadaBFCL/live_simple_48-21-0 (find_beer): brewery='Sierra Nevada', taste=bitter, aroma=hoppy, color=pale, style=lager; defaults omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 069 OUT138

✓

SimpleWeatherChennaiBFCL/live_simple_15-3-11 (get_current_weather): location='Chennai, Tamil Nadu', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 034 OUT133

Tool Calling - Simple: Live Assistants, Weather & Devices (set 12) 3 passed

100.0%

IN15 227 OUT355

✓

SimpleChafodGreensSaladBFCL/live_simple_25-5-3 (ChaFod): TheFod='SALAD' (greens mapped to salad).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 034 OUT75

✓

SimpleFindBeerCreekPorterBFCL/live_simple_49-21-1 (find_beer): brewery='Creek', taste='slightly bitter', aroma=chocolatey, color=dark, style=porter; defaults omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 126 OUT205

✓

SimpleWeatherQuintanaRooBFCL/live_simple_19-3-15 (get_current_weather): location='Quintana Roo, Mexico', unit=celsius.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 067 OUT75

Tool Calling - Simple: Live Assistants, Weather & Devices (set 13) 1 passed1 failed

50.0%

IN12 210 OUT350

✓

SimpleLexListExportsV2BFCL/live_simple_30-8-0 (aws.lexv2_models.list_exports): botId='my-bot-id', botVersion='v2', sortBy=ASC or omitted, others omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 842 OUT119

×

SimpleWeatherLangSonBFCL/live_simple_16-3-12 (get_current_weather): location='Lang Son, Vietnam', unit=fahrenheit or omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 368 OUT231

Tool Calling - Simple: Live Assistants, Weather & Devices (set 14) 2 passed

100.0%

IN9 608 OUT246

✓

SimpleLexListExportsDescBFCL/live_simple_31-8-1 (aws.lexv2_models.list_exports): botId='B12345', botVersion='v1', sortBy=DESC, others omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 846 OUT110

✓

SimpleWeatherBostonBFCL/live_simple_17-3-13 (get_current_weather): location='Boston, MA', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 762 OUT136

Tool Calling - Simple: Live Assistants, Weather & Devices (set 15) 2 passed

100.0%

IN8 722 OUT215

✓

SimpleGetUserInfoBlackBFCL/live_simple_0-0-0 (get_user_info): user_id=7890, special='black'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 369 OUT95

✓

SimpleWeatherMoscowBFCL/live_simple_18-3-14 (get_current_weather): location='Moscow, Russia', unit=celsius.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 353 OUT120

Tool Calling - Simple: Live Assistants, Weather & Devices (set 16) 3 passed

100.0%

IN14 609 OUT659

✓

SimpleFetchWeatherCoordsBFCL/live_simple_39-16-0 (fetch_weather_data): latitude=37.8651, longitude=-119.5383; url/units/language defaults omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 883 OUT274

✓

SimpleGithubStarGorillaBFCL/live_simple_1-1-0 (github_star): repos='ShishirPatil/gorilla,gorilla-llm/gorilla-cli', aligned=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 885 OUT119

✓

SimpleWeatherBangkokBFCL/live_simple_14-3-10 (get_current_weather): location='Bangkok, Thailand', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 841 OUT266

Tool Calling - Simple: Live Assistants, Weather & Devices (set 17) 3 passed

100.0%

IN13 894 OUT253

✓

SimpleCarbonIntensityGBBFCL/live_simple_50-22-0 (get_latest_carbon_intensity): zone='Great Britain' or 'GB'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 594 OUT83

✓

SimpleMultiplyThreeTwoBFCL/live_simple_47-20-0 (multiply): a=3, b=2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 596 OUT63

✓

SimpleWeatherOpenMeteoMariposaBFCL/live_simple_38-15-0 (get_current_weather, open-meteo variant): location='Mariposa, CA', unit=fahrenheit; url default omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 704 OUT107

Tool Calling - Simple: Live Assistants, Weather & Devices (set 2) 3 passed

100.0%

IN14 703 OUT242

✓

SimpleChangeFoodChickenSpicyBFCL/live_simple_21-4-1 (change_food): food_item='chicken dish', modification_request='extra spicy'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 920 OUT89

✓

SimpleThinqAirDryMidBFCL/live_simple_41-17-1 (ThinQ_Connect): AIR_DRY mode, MID wind (or omitted), all else optional. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. windStrength broadened (mid|medium) for "medium wind strength"; air-clean off-state accepts off/power_off.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 893 OUT78

✓

SimpleWeatherDivinopolisBFCL/live_simple_5-3-1 (get_current_weather): location='Divinopolis, MG' (verbatim), unit=fahrenheit or omitted. Prompt is Portuguese, copied verbatim.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 890 OUT75

Tool Calling - Simple: Live Assistants, Weather & Devices (set 3) 3 passed

100.0%

IN14 706 OUT319

✓

SimpleParseAnswerNoDefineBFCL/live_simple_36-13-0 (parseAnswer): answer='The text does not define logistic regression, therefore I cannot answer this question.'1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 895 OUT113

✓

SimpleThinqKoreanTarget22BFCL/live_simple_42-17-2 (ThinQ_Connect): coolTargetTemperature=22 required; mode COOL and wind MID optional. Prompt is Korean, copied verbatim. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. Prompt is dehumidify (제습) so GT's optional airConJobMode=COOL is a source oddity — accept cool/dehumidify/air_dry (optional). windStrength MID broadened (mid|medium). The "22도" target-temp is accepted in either coolTargetTemperature or targetTemperature (the two are genuinely ambiguous for "target temperature").1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 924 OUT120

✓

SimpleWeatherSanFranciscoBFCL/live_simple_6-3-2 (get_current_weather): location='San Francisco, CA', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 887 OUT86

Tool Calling - Simple: Live Assistants, Weather & Devices (set 4) 3 passed

100.0%

IN14 591 OUT251

✓

SimpleParseAnswerNoDefineValidityBFCL/live_simple_37-14-0 (parseAnswer): answer='The text does not define logistic regression, therefore I cannot answer this question.'1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 877 OUT71

✓

SimpleThinqCoolHighWindBFCL/live_simple_43-17-3 (ThinQ_Connect): HIGH wind required; COOL mode and coolTargetTemperature=24 optional. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. air-clean off-state broadened (power_off|off) — the model files the unrequested clean-mode as "off".1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 872 OUT90

✓

SimpleWeatherRigaBFCL/live_simple_7-3-3 (get_current_weather): location='Riga, Latvia'; unit fahrenheit/celsius/omitted all accepted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 842 OUT90

Tool Calling - Simple: Live Assistants, Weather & Devices (set 5) 3 passed

100.0%

IN15 204 OUT254

✓

SimpleThinqTurnedOnCoolBFCL/live_simple_44-18-0 (ThinQ_Connect): airConOperationMode=POWER_ON required; COOL currentJobMode optional. NOTE: tool flattened to nullable top-level params (see struct comment) so a valid partial call is one tool call, not a forced retry. windStrength/airClean off-state broadened to synonyms (mid|medium, stop|off|power_off).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 045 OUT74

✓

SimpleUberComfortAddisonBFCL/live_simple_2-2-0 (uber.ride): loc='2020 Addison Street, Berkeley, CA, USA', type=comfort, time=600.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 107 OUT111

✓

SimpleWeatherLondonBFCL/live_simple_8-3-4 (get_current_weather): location='London, UK' or 'London, England'; unit fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 052 OUT69

Tool Calling - Simple: Live Assistants, Weather & Devices (set 6) 3 passed

100.0%

IN15 251 OUT349

✓

SimpleThinqCoolAirCleanStartBFCL/live_simple_45-18-1 (ThinQ_Connect): airCleanOperationMode=START, airConOperationMode=POWER_ON, coolTargetTemperature=20. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. airCleanOperationMode "air cleaning on" broadened (start|on|power_on). airConOperationMode=POWER_ON and coolTargetTemperature=20 kept strict — both fair GT. EXPECTED MODEL MISS: 27B gets job-mode + air-clean START + temp, but omits airConOperationMode=POWER_ON.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 109 OUT105

✓

SimpleUberPlusBakerBFCL/live_simple_3-2-1 (uber.ride): loc='221B Baker Street, Berkeley, CA, USA', type=plus, time=600.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 109 OUT128

✓

SimpleWeatherHyderabadBFCL/live_simple_9-3-5 (get_current_weather): location='Hyderabad, India', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 033 OUT116

Tool Calling - Simple: Live Assistants, Weather & Devices (set 7) 3 passed

100.0%

IN15 256 OUT451

✓

SimpleThinqFlatCoolCleanBFCL/live_simple_46-19-0 (ThinQ_Connect, flat params): COOL, MID, monitoring on, clean START, POWER_ON, power-save off, target 24C.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 148 OUT161

✓

SimpleUberPlusVietnameseBFCL/live_simple_26-6-0 (uber.ride): loc Vietnamese/English form, type=plus, time=10 (minutes). Prompt is Vietnamese, copied verbatim.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 071 OUT132

✓

SimpleWeatherYosemiteMariposaBFCL/live_simple_10-3-6 (get_current_weather): location='Mariposa, CA', unit=celsius.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 037 OUT158

Tool Calling - Simple: Live Assistants, Weather & Devices (set 8) 2 passed1 failed

66.7%

IN15 093 OUT343

×

SimpleThinqCoolScheduledBFCL/live_simple_51-23-0 (ThinQ_Connect): COOL, HIGH wind, POWER_ON, power-save on, target 22C, start in 1h30m. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. airConOperationMode=POWER_ON kept strict ("Start in 1h30m" schedules a power-on). EXPECTED MODEL MISS: 27B gets job-mode COOL + wind + power-save + temp + schedule, but omits airConOperationMode=POWER_ON.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.airConOperationMode?.ToLower()).AnyOf(["power_on"])

Failed

IN5 100 OUT143

✓

SimpleUberEatPitadaBFCL/live_simple_27-7-0 (uber.eat.order): restaurant='uber pitada', items=[burgers, chicken wings], quantities=[5, 6].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 024 OUT124

✓

SimpleWeatherNaplesBFCL/live_simple_11-3-7 (get_current_weather): location='Naples, FL' or 'Naples, Florida'; unit fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 969 OUT76

Tool Calling - Simple: Live Assistants, Weather & Devices (set 9) 3 passed

100.0%

IN14 062 OUT410

✓

SimpleChafodWhopperBurgerBFCL/live_simple_22-5-0 (ChaFod): TheFod='BURGER' (Whopper mapped to burger).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 643 OUT73

✓

SimpleUberEatKfcChineseBFCL/live_simple_28-7-1 (uber.eat.order): KFC order, 4 items, quantities [10,50,30,90]. Prompt is Chinese, copied verbatim.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 717 OUT160

✓

SimpleWeatherNewYorkBFCL/live_simple_12-3-8 (get_current_weather): location='New York, NY', unit=celsius.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 702 OUT177

Tool Calling - Simple: Live Media, Weather & Data (set 2) 9 passed1 failed

90.0%

IN76 120 OUT2 351

✓

SimpleCreditQuoteAutoCreditoBFCL/live_simple_67-31-0 (obtener_cotizacion_de_creditos): monto=1000000, plazo=12, producto='auto', año=2024, enganche=0.2.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 700 OUT276

✓

SimpleCurrentLocDmsBFCL/live_simple_87-48-0 (get_current_loc): coordinate_format='DMS', include_altitude=true, timeout default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 587 OUT891

✓

SimpleGetMoviesMumbaiTodayBFCL/live_simple_59-28-0 (get_movies): city='Mumbai', optionals default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 555 OUT452

✓

SimplePlayArtistKaarijaTracksBFCL/live_simple_93-54-0 (play_artist): artist_name='Käärijä'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 570 OUT70

✓

SimpleRetrieveAnalysisSuperfancyBFCL/live_simple_69-33-0 (analysis_api.AnalysisApi.retrieve_analysis): project/component/vulnerability UUIDs.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 685 OUT202

✓

SimpleSensorAlertsPerPage10BFCL/live_simple_70-34-0 (get_sensor_alerts): perPage=10, sensorSerial='Q3CC-CRT3-SZ2G', rest omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 633 OUT105

✓

SimpleSitefinityNewsItemJulian12BFCL/live_simple_81-42-0 (sitefinity_create_contentitem): ContentItem='NewsItem', Title='Julian is testing12', rest omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 636 OUT84

✓

SimpleTodoAddShopping9pmB25BFCL/live_simple_55-25-1 (todo): type='add', content='go for shopping at 9 pm'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 579 OUT75

×

SimpleTodoDeleteTodoRandomBFCL/live_simple_54-25-0 (todo): type='delete', content='todo random'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.content?.ToLower()).AnyOf(["todo random"])

Failed

IN7 581 OUT69

✓

SimpleWeatherGetLondonUkBFCL/live_simple_77-38-0 (weather.get): city='London', country='GB' (from UK), units/forecast default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 594 OUT127

Tool Calling - Simple: Live Media, Weather & Data (set 3) 9 passed1 failed

90.0%

IN71 650 OUT1 191

×

SimpleExtractMillennialsAppleBFCL/live_simple_71-35-0 (extract_parameters_v1): demographics=[millennials], targets=[brand:Apple], metrics=[view], min_date=2022-07-01. NOTE: targets @param clarified to '<attribute>:<entity>' order (was ambiguous 'target:target_type', which made the model emit the reversed 'Apple:brand'); GT 'brand:Apple' kept strict.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: args.metrics.Count == 1 Values: args.metrics.Count = 0

Failed

IN7 177 OUT239

✓

SimpleFetchPersonId4BFCL/live_simple_91-52-0 (fetchPerson): id=4.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 141 OUT250

✓

SimpleLogFoodChaiTeaBFCL/live_simple_88-49-0 (log_food): food_name='chai tea', portion_amount=16.0, portion_unit='ounces', meal_name='snack'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 209 OUT93

✓

SimplePlaySongChaChaChaBFCL/live_simple_94-55-0 (play_song): artist_name='Käärijä', song_name='Cha Cha Cha'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 159 OUT78

✓

SimpleSendEmailAndyForecastBFCL/live_simple_78-39-0 (send_email): to='andy@gorilla.ai', subject='Sales Forecast Request', body set, cc/bcc/attachments omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 207 OUT97

✓

SimpleSitefinityNewsJulianTestingBFCL/live_simple_82-43-0 (sitefinity_create_contentitem): ContentItem='News', Title='Julian is testing', rest omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 175 OUT80

✓

SimpleTemperatureParisFranceBFCL/live_simple_90-51-0 (temperature): location='Paris, France', units/time default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 133 OUT96

✓

SimpleTodoManagerAddSleepBFCL/live_simple_57-26-1 (todo_manager.handle_action): type='add', content='go to sleep at 9 pm'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 146 OUT78

✓

SimpleTodoManagerDeleteGymBFCL/live_simple_56-26-0 (todo_manager.handle_action): type='delete', content='go to gym'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 136 OUT70

✓

SimpleTranslateEnglishFrenchBFCL/live_simple_76-37-0 (language_translator.translate): source='en', target='fr', text='What is your name?'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 167 OUT110

Tool Calling - Simple: Live Media, Weather & Data (set 4) 10 passed

100.0%

IN53 342 OUT1 077

✓

SimpleOpenMeteoBerkeleyBFCL/live_simple_85-46-0 (open_meteo_api.fetch_weather_data): coordinates=[37.8715,-122.273], units default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 348 OUT245

✓

SimpleSearchHotelsSanDiegoBFCL/live_simple_79-40-0 (search_hotels): location='San Diego, CA'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 388 OUT87

✓

SimpleSitefinityNewsSleepProductivityBFCL/live_simple_83-44-0 (sitefinity_create_contentitem): full news article with all metadata.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 482 OUT215

✓

SimpleSortArrayAscendingBFCL/live_simple_86-47-0 (sort_array): array=[1,54,3,1,76,2343,21], reverse default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 364 OUT111

✓

SimpleTodoAddGoToGoaBFCL/live_simple_64-29-4 (todo): type='add', content='go to Goa'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 285 OUT68

✓

SimpleTodoAddGymTomorrowBFCL/live_simple_61-29-1 (todo): type='add', content='go to gym tomorrow.'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 284 OUT68

✓

SimpleTodoAddHiCharlieBFCL/live_simple_65-29-5 (todo): type='add', content='Hi charlie'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 295 OUT66

✓

SimpleTodoAddSleep9pmBFCL/live_simple_63-29-3 (todo): type='add', content='go to sleep at 9 pm'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 291 OUT74

✓

SimpleTodoCompleteFinancialReportBFCL/live_simple_60-29-0 (todo): type='complete', content='Submit monthly financial report'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 308 OUT74

✓

SimpleTodoDeleteRaviBFCL/live_simple_62-29-2 (todo): type='delete', content='ravi'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 297 OUT69

Tool Calling - Simple: Live Movies, Music & Booking (set 1) 3 passed

100.0%

IN14 609 OUT246

✓

SimpleMoviesTarantinoWhitakerBFCL/live_simple_208-117-0 (Movies_3_FindMovies): directed_by='Quentin Tarantino', cast='Duane Whitaker', genre dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 890 OUT100

✓

SimplePlaySongRoarBFCL/live_simple_230-121-0 (play_song): song_id='wjeiruhuq_roar', artist='Katy Perry', quality='high', device_id omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 887 OUT91

✓

SimpleSetVolume20BFCL/live_simple_242-126-0 (set_volume): volume=20.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 832 OUT55

Tool Calling - Simple: Live Movies, Music & Booking (set 10) 3 passed

100.0%

IN15 611 OUT505

✓

SimpleHandoverPackageArrivalBFCL/live_simple_257-137-1 (answer_question): statement=package-arrival-time question, urgency/language default. System instructs HANDOVER on question. NOTE: no collider loop observed (single clean call, sibling tests pass); statement is free text -> substring-intent of the key phrase (model drops "Can you tell me").1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 204 OUT90

✓

SimpleMoviesNiteshDirectorBFCL/live_simple_217-117-9 (Movies_3_FindMovies): directed_by='Nitesh Tiwari', genre/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 190 OUT295

✓

SimpleTtsChineseMaleBFCL/live_simple_227-118-1 (text_to_speech.convert): text=Chinese phrase, language='zh-CN', gender='male', format='wav', speed default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 217 OUT120

Tool Calling - Simple: Live Movies, Music & Booking (set 11) 3 passed

100.0%

IN14 968 OUT251

✓

SimpleMoviesColaizzoWatkinsBFCL/live_simple_218-117-10 (Movies_3_FindMovies): directed_by='Paul Downs Colaizzo', cast='Michaela Watkins', genre dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 049 OUT104

✓

SimpleRequestsGetBuProductsBFCL/live_simple_229-120-0 (requests.get): anchor='user'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 958 OUT78

✓

SimpleTtsPrettyGirlDefaultBFCL/live_simple_228-119-0 (text_to_speech.convert): text='I am a pretty girl', language/gender/speed default.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 961 OUT69

Tool Calling - Simple: Live Movies, Music & Booking (set 12) 3 passed

100.0%

IN14 674 OUT298

✓

SimpleBookFlightParisBFCL/live_simple_233-123-0 (book_flight): departure_location='Paris, France', departure_time='12/03/2023 15:00', return_time omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 927 OUT96

✓

SimpleGetVersionBFCL/live_simple_247-129-0 (version_api.VersionApi.get_version): no arguments.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 839 OUT109

✓

SimpleMoviesSimonCurtisDramaBFCL/live_simple_219-117-11 (Movies_3_FindMovies): directed_by='Simon Curtis', genre='Drama', cast='McKinley Belcher III'.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 908 OUT93

Tool Calling - Simple: Live Movies, Music & Booking (set 13) 3 passed

100.0%

IN15 220 OUT576

✓

SimpleAclRetrieveProjectsBFCL/live_simple_248-130-0 (acl_api.AclApi.retrieve_projects): uuid='12', excludeInactive=true, onlyRoot=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 058 OUT355

✓

SimpleBookFlightNewYorkToParisBFCL/live_simple_234-123-1 (book_flight): departure_location='New York, NY', departure_time='20/06/2022 17:00', return_time omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 108 OUT122

✓

SimpleMoviesMackendrickThrillerBFCL/live_simple_220-117-12 (Movies_3_FindMovies): directed_by='Alexander Mackendrick', genre='Thriller', cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 054 OUT99

Tool Calling - Simple: Live Movies, Music & Booking (set 14) 3 passed

100.0%

IN14 848 OUT518

✓

SimpleBookFlightJfkBFCL/live_simple_245-127-0 (book_flight): departure_location='JFK', departure_date='16/03/2024', departure_time='14:00'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 999 OUT140

✓

SimpleChatWithBunnyBFCL/live_simple_249-131-0 (chat_with_friend): user_message='Hey bunny, How are you doing'. NOTE: no collider loop observed (single clean call, sibling tests pass); user_message is a free-text passthrough -> substring-intent (model may drop the "Hey bunny," greeting).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 911 OUT69

✓

SimpleMoviesJamesGrayBFCL/live_simple_221-117-13 (Movies_3_FindMovies): directed_by='James Gray', genre/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 938 OUT309

Tool Calling - Simple: Live Movies, Music & Booking (set 15) 2 passed

100.0%

IN9 308 OUT319

✓

SimpleBookFlightNewYorkMarchBFCL/live_simple_246-128-0 (book_flight): departure_location='New York, NY', departure_time='15/03/2024'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 646 OUT132

✓

SimpleMoviesVanOrmanBrownBFCL/live_simple_222-117-14 (Movies_3_FindMovies): directed_by='Thurop Van Orman', cast='Sterling K. Brown', genre dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 662 OUT187

Tool Calling - Simple: Live Movies, Music & Booking (set 16) 1 passed1 skipped

100.0%

IN4 622 OUT200

✓

SimpleMoviesImaginativeFantasyBFCL/live_simple_223-117-15 (Movies_3_FindMovies): genre='Fantasy' ('Imaginative fiction'), director/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 622 OUT200

–

SimpleRescheduleGymSessionBFCL/live_simple_231-122-0 (reschedule_event): event_identifier='456123', new_datetime='2022-10-30T16:30:00Z'.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: source flaw: relative-date arithmetic is inconsistent — prompt states 'Monday 2022-10-20' (that date is actually a Thursday) and the GT 'next Thursday' resolves to 2022-10-30 (a Sunday), so the exact new_datetime cannot be derived deterministically.

Skipped

IN0 OUT0

Tool Calling - Simple: Live Movies, Music & Booking (set 17) 2 passed

100.0%

IN9 274 OUT354

✓

SimpleMoviesTimBurtonDeppBFCL/live_simple_224-117-16 (Movies_3_FindMovies): directed_by='Tim Burton', genre='Offbeat', cast='Johnny Depp'.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 593 OUT248

✓

SimpleRescheduleMedAppointmentBFCL/live_simple_232-122-1 (reschedule_event): event_identifier='med123', new_datetime='2021-04-15T13:45:00Z'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 681 OUT106

Tool Calling - Simple: Live Movies, Music & Booking (set 18) 2 passed

100.0%

IN8 971 OUT257

✓

SimpleMoviesKirillComedyDramaBFCL/live_simple_225-117-17 (Movies_3_FindMovies): directed_by='Kirill Mikhanovsky', genre='Comedy-drama', cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 525 OUT201

✓

SimpleSetVolume70BFCL/live_simple_244-126-2 (set_volume): volume=70.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 446 OUT56

Tool Calling - Simple: Live Movies, Music & Booking (set 2) 3 passed

100.0%

IN14 654 OUT247

✓

SimpleMoviesLeitchTuisanoBFCL/live_simple_209-117-1 (Movies_3_FindMovies): directed_by='David Leitch', cast='Lori Pelenise Tuisano', genre dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 915 OUT121

✓

SimpleSetVolume30BFCL/live_simple_243-126-1 (set_volume): volume=30.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 868 OUT57

✓

SimpleSpotifyFriendsMarshmelloBFCL/live_simple_235-124-0 (play_spotify_song): query='track:Friends artist:Marshmello', shuffle/volume default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 871 OUT69

Tool Calling - Simple: Live Movies, Music & Booking (set 3) 3 passed

100.0%

IN15 239 OUT387

✓

SimpleFlightStatusIndigo6E123BFCL/live_simple_251-133-0 (flight.status.check): flight_id='6E123', passenger_name='Karan', ticket_number='IND4567'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 106 OUT116

✓

SimpleMoviesDramaBFCL/live_simple_210-117-2 (Movies_3_FindMovies): genre='Drama', director/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 076 OUT195

✓

SimpleSpotifyDilNu124BFCL/live_simple_236-124-1 (play_spotify_song): query='track:Dil Nu artist:Maninder Buttar', shuffle/volume default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 057 OUT76

Tool Calling - Simple: Live Movies, Music & Booking (set 4) 1 passed2 failed

33.3%

IN15 222 OUT294

✓

SimpleFlightStatusIndigoAirlineBFCL/live_simple_252-134-0 (flight.status.check): airlines='Indigo', person_name='Karan', ticket_number='IND4567'. NOTE: tool doc says "if only one name mentioned ... use LNU", so "Karan LNU" is equivalent-correct per the documented convention -> accept both.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 100 OUT130

×

SimpleMoviesComedyCordenBFCL/live_simple_211-117-3 (Movies_3_FindMovies): genre='Comedy' (prompt 'Comic'), cast='James Corden', director dontcare. NOTE: args are correct; single-call contract stays strict. The 27B occasionally emits a 2nd call (over-call) -> legit intermittent fail (kept as signal).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.genre?.ToLower()).AnyOf(["comedy"])

Failed

IN5 086 OUT89

×

SimpleSpotifyWreckingBallBFCL/live_simple_237-125-0 (play_spotify_song): query='track:Wrecking Ball artist:Miley Cyrus', device_id/play default. NOTE: GT accepts only the structured 'track:.. artist:..' form (kept strict). The 27B occasionally drops the format (e.g. "wrecking ball Miley Cyrus") -> legit intermittent miss (kept as signal).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.query?.ToLower()).AnyOf(["track:wrecking ball artist:miley cyrus"])

Failed

IN5 036 OUT75

Tool Calling - Simple: Live Movies, Music & Booking (set 5) 3 passed

100.0%

IN15 152 OUT518

✓

SimpleMoviesComedyEdgarWrightBFCL/live_simple_212-117-4 (Movies_3_FindMovies): directed_by='Edgar Wright', genre='Comedy', cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 041 OUT322

✓

SimpleSpotifyDilNu125BFCL/live_simple_238-125-1 (play_spotify_song): query='track:Dil Nu artist:Maninder Buttar', device_id/play default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 041 OUT76

✓

SimpleUberEatsBigMacBFCL/live_simple_250-132-0 (uber.eat.order): restaurants='McDonald's', items=[Big Mac, McFlurry], quantities=[2,1].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 070 OUT120

Tool Calling - Simple: Live Movies, Music & Booking (set 6) 3 passed

100.0%

IN15 172 OUT328

✓

SimpleMoviesOffbeatTimBurtonBFCL/live_simple_213-117-5 (Movies_3_FindMovies): directed_by='Tim Burton', genre='Offbeat', cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 060 OUT133

✓

SimpleSpotifyShapeOfYouBFCL/live_simple_239-125-2 (play_spotify_song): query='track:Shape of You artist:Ed Sheeran' or 'Shape of You'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 040 OUT82

✓

SimpleUberEatsBurgersWingsBFCL/live_simple_253-135-0 (uber.eat.order): restaurants='McDonald's', items=[burgers, chicken wings], quantities=[5,6].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 072 OUT113

Tool Calling - Simple: Live Movies, Music & Booking (set 7) 3 passed

100.0%

IN14 981 OUT484

✓

SimpleMoviesComicNiteshTiwariBFCL/live_simple_214-117-6 (Movies_3_FindMovies): directed_by='Nitesh Tiwari', genre='Comedy' or 'Comedy-drama', cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 052 OUT320

✓

SimpleSpotifyBabySharkBFCL/live_simple_240-125-3 (play_spotify_song): query='Baby Shark' (artist unknown).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 953 OUT67

✓

SimpleUserMandatesPendingBFCL/live_simple_254-136-0 (user.mandates): user_id='U123', status='pending'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 976 OUT97

Tool Calling - Simple: Live Movies, Music & Booking (set 8) 3 passed

100.0%

IN14 930 OUT455

✓

SimpleMoviesFantasyTonightBFCL/live_simple_215-117-7 (Movies_3_FindMovies): genre='Fantasy', director/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 992 OUT285

✓

SimpleSpotifyJohnnyPapaBFCL/live_simple_241-125-4 (play_spotify_song): query='Johnny Johnny Yes Papa' (artist unknown).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 949 OUT70

✓

SimpleUserMandatesActiveBFCL/live_simple_255-136-1 (user.mandates): user_id='U123456', status='active' or omitted (default).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 989 OUT100

Tool Calling - Simple: Live Movies, Music & Booking (set 9) 2 passed1 failed

66.7%

IN15 570 OUT318

×

SimpleHandoverComplaintStatusBFCL/live_simple_256-137-0 (answer_question): statement=user complaint-status question, urgency/language default. System instructs HANDOVER on question.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.statement?.ToLower()).AnyOf(["i have had submit a compaint and i like to check its status"])

Failed

IN5 203 OUT98

✓

SimpleMoviesActionLeitchKingBFCL/live_simple_216-117-8 (Movies_3_FindMovies): directed_by='David Leitch', genre='Action', cast='Alex King'.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 180 OUT100

✓

SimpleTtsPrettyGirlSpanishBFCL/live_simple_226-118-0 (text_to_speech.convert): text='I am a pretty girl' or Spanish translation, language='es-ES', gender/format/speed default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 187 OUT120

Tool Calling - Simple: Live Services, Weather & Commands (set 1) 4 passed

100.0%

IN18 950 OUT404

✓

SimpleExtractChesterJaneBFCL/live_simple_189-114-0 (extractor.extract_information): [Chester,42],[Jane,43] (Jane a year older).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 784 OUT158

✓

SimpleServiceIdIroningUnitIntBFCL/live_simple_174-100-0 (get_service_id): ironing service -> service_id=2, unit omitted/1. Thai prompt.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 731 OUT69

✓

SimpleWeatherNewYorkBFCL/live_simple_193-116-1 (Weather_1_GetWeather): city=New York, date=2023-03-08.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 717 OUT87

✓

SimpleWeatherPacificaBFCL/live_simple_192-116-0 (Weather_1_GetWeather): city=Pacifica, date=2023-04-11.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 718 OUT90

Tool Calling - Simple: Live Services, Weather & Commands (set 10) 7 passed2 failed

77.8%

IN41 509 OUT892

✓

SimpleCmdDockerPsBFCL/live_simple_172-99-6 (cmd_controller.execute): is Docker running -> 'docker ps'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 577 OUT75

✓

SimpleCmdDockerStartBFCL/live_simple_168-99-2 (cmd_controller.execute): start Docker -> 'docker start'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 577 OUT56

✓

SimpleCmdEchoDateBFCL/live_simple_171-99-5 (cmd_controller.execute): today's date -> 'echo %date%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 586 OUT71

✓

SimpleCmdEchoTimeBFCL/live_simple_166-99-0 (cmd_controller.execute): current time -> 'echo %time%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 598 OUT62

✓

SimpleCmdListCDriveBFCL/live_simple_170-99-4 (cmd_controller.execute): list C drive -> 'dir C:\'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 567 OUT106

×

SimpleCmdRunPythonScriptBFCL/live_simple_169-99-3 (cmd_controller.execute): run python script with venv interpreter.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.command?.ToLower()).AnyOf(["d:/playground/pc_contoller/env/scripts/python.exe d:/playground/pc_contoller/test.py"])

Failed

IN4 663 OUT114

✓

SimpleCmdStartCalcBFCL/live_simple_173-99-7 (cmd_controller.execute): open calculator -> 'start calc'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 579 OUT55

✓

SimpleCmdTasklistCodeBFCL/live_simple_167-99-1 (cmd_controller.execute): is VS Code running -> tasklist filter.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 610 OUT96

×

SimpleProvidersChonburiDateRangeBFCL/live_simple_186-111-0 (get_service_providers): ironing, Chonburi, rating>=4.5, 12/03/2024 16:00-18:00.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.start_available_date?.ToLower()).AnyOf(["2024-03-12 16:00:00", "2024-03-12 16:00"])

Failed

IN4 752 OUT257

Tool Calling - Simple: Live Services, Weather & Commands (set 11) 4 passed

100.0%

IN19 734 OUT777

✓

SimpleItemsCircleLogosBFCL/live_simple_162-96-1 (get_items): 4 circle-logo copyright items.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 830 OUT149

✓

SimpleItemsLegoMovieBFCL/live_simple_163-96-2 (get_items): 7 Lego-movie copyright items.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 925 OUT103

✓

SimpleItemsSuperheroLogosBFCL/live_simple_161-96-0 (get_items): 12 superhero-logo copyright items.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 087 OUT266

✓

SimpleProvidersChonburiNoLateBFCL/live_simple_187-112-0 (get_service_providers): ironing, Chonburi, rating>=4.5, 12/03/2024, never late. NOTE: the prompt gives a time range "16:00 - 18:00", so end_available_date is expected; the 27B occasionally omits it (~1/3 of runs) -> a legit model-reliability miss, not an assertion flaw.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 892 OUT259

Tool Calling - Simple: Live Services, Weather & Commands (set 12) 2 passed

100.0%

IN9 442 OUT209

✓

SimpleCopyrightCocaColaBFCL/live_simple_164-97-0 (get_copyright_info): Coca-Cola logo, The Coca-Cola Company, confidence 100.0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 867 OUT104

✓

SimpleProfessionalCleaningRating2BFCL/live_simple_183-108-0 (getDataForProfessional): cleaning in Bangkok, rating>=2.0 -> avg_rating=2.0, province_id=1, service_id=1.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 575 OUT105

Tool Calling - Simple: Live Services, Weather & Commands (set 13) 3 passed

100.0%

IN13 709 OUT455

✓

SimpleMpnSpecs3pak7BFCL/live_simple_191-115-1 (raptor.mpn.specs): identifier=3pak7, MPN, include_images omitted/false.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 570 OUT106

✓

SimpleMpnSpecsAchdImagesBFCL/live_simple_190-115-0 (raptor.mpn.specs): identifier=ACHD, MPN, include_images=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 573 OUT111

✓

SimpleProfessionalCleaningRating4BFCL/live_simple_184-109-0 (getDataForProfessional): cleaning in Bangkok, rating>4 -> service_id=1, province_id=1, rating=4.0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 566 OUT238

Tool Calling - Simple: Live Services, Weather & Commands (set 14) 2 passed

100.0%

IN9 779 OUT317

✓

SimpleExtractChineseNamesBFCL/live_simple_165-98-0 (extractor.extract_information): Chinese text -> [李雷,18],[李丽,21] (sister 3 yrs older).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 896 OUT177

✓

SimpleProfessionalIroningChiangMaiBFCL/live_simple_188-113-0 (getDataForProfessional): ironing in Chiang Mai, Sankampang/Ton Pao, rating 4.5.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 883 OUT140

Tool Calling - Simple: Live Services, Weather & Commands (set 2) 3 passed

100.0%

IN13 122 OUT271

✓

SimpleServiceIdIroningUnitSessionBFCL/live_simple_175-101-0 (get_service_id): ironing -> service_id=2, unit omitted/session. Thai prompt.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 357 OUT85

✓

SimpleWeatherMartinezBFCL/live_simple_194-116-2 (Weather_1_GetWeather): city=Martinez, date=2023-04-25.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 391 OUT97

✓

SimpleWeatherPaloAltoBFCL/live_simple_195-116-3 (Weather_1_GetWeather): city=Palo Alto, date=2023-04-25.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 374 OUT89

Tool Calling - Simple: Live Services, Weather & Commands (set 3) 2 passed1 failed

66.7%

IN13 084 OUT250

✓

SimpleServiceIdIroningUnitQtyBFCL/live_simple_176-102-0 (get_service_id): ironing -> service_id=2, unit omitted/1. Thai prompt.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 343 OUT68

×

SimpleWeatherAlamedaWednesdayBFCL/live_simple_196-116-4 (Weather_1_GetWeather): city=Alameda, upcoming Wednesday (today Mon 2023-04-25) -> 2023-04-27.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.date).AnyOf(["2023-04-27"]) Values: args.date = "2023-04-26"

Failed

IN4 373 OUT92

✓

SimpleWeatherStinsonBeachBFCL/live_simple_197-116-5 (Weather_1_GetWeather): city=Stinson Beach, date=2023-04-05.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 368 OUT90

Tool Calling - Simple: Live Services, Weather & Commands (set 4) 4 passed

100.0%

IN17 419 OUT357

✓

SimpleServiceIdCleaningBFCL/live_simple_177-103-0 (get_service_id): cleaning -> service_id=1, unit omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 329 OUT86

✓

SimpleServiceIdIroningBFCL/live_simple_178-103-1 (get_service_id): ironing -> service_id=2, unit omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 331 OUT78

✓

SimpleWeatherHealdsburgBFCL/live_simple_198-116-6 (Weather_1_GetWeather): city=Healdsburg, date=2023-03-02.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 384 OUT103

✓

SimpleWeatherMarshallBFCL/live_simple_199-116-7 (Weather_1_GetWeather): city=Marshall (prompt 'Marshall, MN'), date=2023-03-05. NOTE: GT city is bare "Marshall"; model faithfully copies the "Marshall, MN" disambiguator from the prompt -> substring-intent (equivalent-correct).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 375 OUT90

Tool Calling - Simple: Live Services, Weather & Commands (set 5) 3 passed

100.0%

IN13 221 OUT267

✓

SimpleServiceIdIroningChiangMaiBFCL/live_simple_179-104-0 (get_service_id): ironing in Chiang Mai -> service_id=2, province_id=2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 378 OUT81

✓

SimpleWeatherCampbellBFCL/live_simple_201-116-9 (Weather_1_GetWeather): city=Campbell, date=2023-03-04.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 435 OUT98

✓

SimpleWeatherFremontBFCL/live_simple_200-116-8 (Weather_1_GetWeather): city=Fremont, date=2023-03-01.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 408 OUT88

Tool Calling - Simple: Live Services, Weather & Commands (set 6) 3 passed

100.0%

IN13 408 OUT375

✓

SimpleServiceIdIroningBangkokRatingBFCL/live_simple_180-105-0 (get_service_id): ironing in Bangkok, rating>=4 -> service_id=2, province_id=1, rating=4.0.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 472 OUT166

✓

SimpleWeatherFosterCityBFCL/live_simple_202-116-10 (Weather_1_GetWeather): city=Foster City, date=2023-04-25.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 465 OUT111

✓

SimpleWeatherWashingtonBFCL/live_simple_203-116-11 (Weather_1_GetWeather): city=Washington (prompt 'Washington, DC'), date=2023-03-01. NOTE: GT city is bare "Washington"; model faithfully copies the "Washington, DC" disambiguator from the prompt -> substring-intent (equivalent-correct).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 471 OUT98

Tool Calling - Simple: Live Services, Weather & Commands (set 7) 3 passed

100.0%

IN13 733 OUT323

✓

SimpleServiceIdIroningChatuchakBFCL/live_simple_181-106-0 (get_service_id): ironing, Bangkok(Chatuchak), rating>=4.5 -> service_id=2, province_id omitted/1, district_name=Chatuchak, rating=4.5.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 589 OUT128

✓

SimpleWeatherBerkeleySaturdayBFCL/live_simple_205-116-13 (Weather_1_GetWeather): city=Berkeley, upcoming Saturday (today Tue 2023-04-25) -> 2023-04-29.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 568 OUT101

✓

SimpleWeatherRutherfordBFCL/live_simple_204-116-12 (Weather_1_GetWeather): city=Rutherford (prompt 'Rutherford, NJ'), date=2023-04-22. NOTE: GT city is bare "Rutherford"; model faithfully copies the "Rutherford, NJ" disambiguator from the prompt -> substring-intent (equivalent-correct).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 576 OUT94

Tool Calling - Simple: Live Services, Weather & Commands (set 8) 3 passed

100.0%

IN13 832 OUT438

✓

SimpleProvidersKhlongSamWaBFCL/live_simple_182-107-0 (get_service_providers): ironing, Bangkok, Khlong Sam Wa / Sai Kong, rating>=4.5.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 643 OUT173

✓

SimpleWeatherLondonBFCL/live_simple_206-116-14 (Weather_1_GetWeather): city=London (prompt 'London, England'), date=2023-03-05.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 595 OUT88

✓

SimpleWeatherSacramentoBFCL/live_simple_207-116-15 (Weather_1_GetWeather): city=Sacramento, date=2023-04-22.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 594 OUT177

Tool Calling - Simple: Live Services, Weather & Commands (set 9) 6 passed

100.0%

IN27 860 OUT653

✓

SimpleCmdEchoTimeUnitBFCL/live_simple_157-95-14 (cmd_controller.execute): current time -> 'echo %time%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 623 OUT173

✓

SimpleCmdListCDriveUnitBFCL/live_simple_156-95-13 (cmd_controller.execute): list C drive -> 'dir C:\'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 642 OUT77

✓

SimpleCmdStartCameraBFCL/live_simple_160-95-17 (cmd_controller.execute): open camera -> 'start microsoft.windows.camera:'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 618 OUT57

✓

SimpleCmdTaskkillCodeBFCL/live_simple_158-95-15 (cmd_controller.execute): kill VS Code -> 'taskkill /IM code.exe /F'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 618 OUT69

✓

SimpleCmdTimeout10BFCL/live_simple_159-95-16 (cmd_controller.execute): timer 10s -> 'timeout 10'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 638 OUT70

✓

SimpleProvidersChonburiTomorrowBFCL/live_simple_185-110-0 (get_service_providers): ironing, Chonburi, rating>=4.5, tomorrow of 20230929 -> 2023-09-30.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 721 OUT207

Tool Calling - Simple: Live Shell, Classification & APIs (set 1) 3 passed

100.0%

IN13 889 OUT373

✓

SimpleClassifyGreetingsAndLinksBFCL/live_simple_105-62-0 (classify): clean_hello=['hello'], faq_link_accounts_start=['link my account','connect my accounts'].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 601 OUT118

✓

SimpleCmdDockerVersionBFCL/live_simple_141-94-0 (cmd_controller.execute): command='docker --version'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 580 OUT63

✓

SimpleReqNodesPekAciBFCL/live_simple_128-83-0 (requests.get): nodes list url with ip 192.168.11.33, fabricName='PEK-ACI'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 708 OUT192

Tool Calling - Simple: Live Shell, Classification & APIs (set 10) 4 passed

100.0%

IN19 621 OUT384

✓

SimpleCmdShutdownBFCL/live_simple_150-95-7 (cmd_controller.execute): shutdown the pc via 'shutdown /s /t 0'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 870 OUT70

✓

SimpleProcessImageMoondreamBFCL/live_simple_122-78-0 (process_data): image.png + long question + model 'vikhyatk/moondream2'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 982 OUT151

✓

SimpleReqDataLimit50BFCL/live_simple_135-88-0 (requests.get): data endpoint url, params limit=50.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 901 OUT94

✓

SimpleSumNumbersListBFCL/live_simple_116-72-0 (sum_numbers): numbers_list=[133, 34].1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 868 OUT69

Tool Calling - Simple: Live Shell, Classification & APIs (set 11) 4 passed

100.0%

IN20 039 OUT529

✓

SimpleCmdWifiDisconnectBFCL/live_simple_151-95-8 (cmd_controller.execute): disconnect wifi via 'netsh wlan disconnect'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 897 OUT57

✓

SimplePipelineFurnitureImageBFCL/live_simple_123-79-0 (pipeline): analyze thewowstyle image with furniture/minimalist question.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 173 OUT235

✓

SimpleReqTopologySalesAppBFCL/live_simple_136-89-0 (requests.get): topology url ip 192.168.1.1, filter accountName:AcmeCorp AND applicationName:SalesApp.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 054 OUT143

✓

SimpleTemperatureHyderabadBFCL/live_simple_118-74-0 (get_temperature): units=Celsius, location='Hyderabad, India', time omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 915 OUT94

Tool Calling - Simple: Live Shell, Classification & APIs (set 12) 4 passed

100.0%

IN21 328 OUT553

✓

SimpleCmdDirDAndEchoBFCL/live_simple_152-95-9 (cmd_controller.execute): 'dir D:\ && echo testing.txt'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 277 OUT101

✓

SimpleReqAnomaliesSummaryBFCL/live_simple_139-92-0 (requests.get): anomalies summary url ip 192.168.1.1, params fabric='network222'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 403 OUT134

✓

SimpleRescheduleAlice8pmBFCL/live_simple_137-90-0 (reschedule): identifier='Alice-One-one-One', 2023-11-01T20:00:00, Europe/London.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 310 OUT116

✓

SimpleWeatherForecastParisBFCL/live_simple_120-76-0 (weather.forecast): Paris, France 2023-04-03..05, Celsius, defaults omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 338 OUT202

Tool Calling - Simple: Live Shell, Classification & APIs (set 13) 3 passed

100.0%

IN14 722 OUT417

✓

SimpleCmdDelPuttyBFCL/live_simple_153-95-10 (cmd_controller.execute): remove putty shortcut via del with quoted path.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 822 OUT69

✓

SimpleDynamicPricingSaaSBFCL/live_simple_113-69-0 (calculate_dynamic_pricing): geolocation='34.0522, -118.2437', base=100, min=90.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 042 OUT228

✓

SimpleRescheduleBob4pmBFCL/live_simple_138-91-0 (reschedule): identifier='Bob-123', 2023-11-01T16:00:00, Europe/London.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 858 OUT120

Tool Calling - Simple: Live Shell, Classification & APIs (set 14) 3 passed

100.0%

IN14 106 OUT300

✓

SimpleCmdStartYoutubeBFCL/live_simple_154-95-11 (cmd_controller.execute): open youtube rickroll via start command.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 715 OUT86

✓

SimpleGetTicketsCompanyOpenBFCL/live_simple_124-80-0 (get_tickets): customer (company name in prompt), status=open, limit=10.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 700 OUT116

✓

SimpleGreetPartnerJeffBFCL/live_simple_140-93-0 (greet_partner): user='Emily', partner='Jeff', relationship='husband'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 691 OUT98

Tool Calling - Simple: Live Shell, Classification & APIs (set 15) 2 passed1 failed

66.7%

IN14 479 OUT385

✓

SimpleCalculateTaxNewYorkBFCL/live_simple_104-61-2 (calculate_tax): purchase_amount=250, state='NY', all optionals omitted/default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 836 OUT121

✓

SimpleCmdDateBFCL/live_simple_155-95-12 (cmd_controller.execute): what is the date via 'date /t' or 'echo %date%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 809 OUT199

×

SimpleConcatJohnDoeBFCL/live_simple_125-81-0 (concat_strings): strings=['John','Doe'], separator=' '. EXPECTED MODEL MISS: 'combine ... into a full name' requires separator=' ' (John Doe). The 27B tends to omit separator (=> 'JohnDoe'); that is a legit model failure of the measured competence, so the ' ' assertion stays strict.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.separator.AnyOf([" "]) Values: args.separator = null

Failed

IN4 834 OUT65

Tool Calling - Simple: Live Shell, Classification & APIs (set 2) 3 passed

100.0%

IN16 120 OUT982

✓

SimpleCmdDirDesktopBFCL/live_simple_142-94-1 (cmd_controller.execute): command='dir Desktop'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 045 OUT70

✓

SimpleRecordBankNineIntentsBFCL/live_simple_106-63-0 (record): classify 20 bank queries into 9 intents (2 buckets empty).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 967 OUT785

✓

SimpleReqNodesFabEdBFCL/live_simple_129-83-1 (requests.get): nodes list url with ip 3.321.3232.2, fabricName='fab-ed'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 108 OUT127

Tool Calling - Simple: Live Shell, Classification & APIs (set 3) 4 passed

100.0%

IN21 495 OUT676

✓

SimpleCmdDockerPsBFCL/live_simple_143-95-0 (cmd_controller.execute): command='docker ps'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 235 OUT71

✓

SimpleRecordTelephoneHoursAccountsBFCL/live_simple_107-64-0 (record): classify 7 queries into telephone-banking / bank-hours / accounts.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 644 OUT345

✓

SimpleTelemetryGlobalFabricStatusBFCL/live_simple_126-82-0 (telemetry.flowrules.interfaceInfo.get): fabric='Global-Fabric', node=1200, pod=3, ethernet, status.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 303 OUT123

✓

SimpleUpdateProfileEmailAgeBFCL/live_simple_114-70-0 (update_user_profile): user_id=12345, email+age set, bio omitted, notify default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 313 OUT137

Tool Calling - Simple: Live Shell, Classification & APIs (set 4) 3 passed1 failed

75.0%

IN20 874 OUT697

✓

SimpleCmdKillFirefoxBFCL/live_simple_144-95-1 (cmd_controller.execute): close firefox via taskkill.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 085 OUT66

×

SimpleRecordMoneyTransactionsCreditBFCL/live_simple_108-65-0 (record): classify 10 queries into money-movement / transactions / credit-limit / link-accounts. EXPECTED MODEL MISS: the 27B classifies 'show me all outgoing wire transfers' as a transaction-view (get_transactions_start) rather than money_movement_start per GT, so the get_transactions count comes out as 5 instead of 4. GT bucketing kept strict; this is a legit model classification failure.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: args.money_movement_start.Count == 1 Values: args.money_movement_start.Count = 0

Failed

IN5 553 OUT436

✓

SimpleReverseSayHiBFCL/live_simple_117-73-0 (reverse_input): input_value='say hi'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 073 OUT60

✓

SimpleTelemetryTestDeStatusBFCL/live_simple_127-82-1 (telemetry.flowrules.interfaceInfo.get): fabric='test-de', node=5, pod=1, ethernet, status.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 163 OUT135

Tool Calling - Simple: Live Shell, Classification & APIs (set 5) 3 passed1 failed

75.0%

IN20 888 OUT661

×

SimpleCmdDirCDriveBFCL/live_simple_145-95-2 (cmd_controller.execute): list C drive => 'dir C:\'. EXPECTED MODEL MISS: for the terse/typo'd 'LIST C DRIIVE' the 27B currently returns 'dir d:' — the wrong drive. The accepted-command list is broadened with the no-backslash 'dir c:' variant for fairness, but a wrong drive letter is a legit model failure.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: args.command.ToLower().AnyOf(["dir c:", "dir c:\\", "dir c:\\\\"])

Failed

IN5 109 OUT84

✓

SimplePrimeMinistersThreeBFCL/live_simple_119-75-0 (GetPrimeMinisters): countries=[Australia, Canada, India].1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 143 OUT81

✓

SimpleRecordAutoWithdrawIncomeBFCL/live_simple_109-66-0 (record): classify 5 loan/withdraw queries; outofscope empty.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 361 OUT301

✓

SimpleReqInterfaceStatsNetwork1BFCL/live_simple_130-84-0 (requests.get): interfaceInfo stats, ip 192.120.45.67, Network1/node 12/pod 10/ethernet/statistics.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 275 OUT195

Tool Calling - Simple: Live Shell, Classification & APIs (set 6) 4 passed

100.0%

IN21 571 OUT760

✓

SimpleCmdStartHuggingfaceBFCL/live_simple_146-95-3 (cmd_controller.execute): open huggingface via start command.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 317 OUT109

✓

SimpleGenerateSalaryChartBFCL/live_simple_121-77-0 (generate_chart): 5 job labels and salaries in order.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 450 OUT269

✓

SimpleRecordAtmLocationOnlyBFCL/live_simple_110-67-0 (record): single query 'Need ATM location' -> atm_finder_start only.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 295 OUT91

✓

SimpleReqInterfaceSpeedNetwork1BFCL/live_simple_131-84-1 (requests.get): interfaceInfo Speed, ip 192.120.45.67, Network1/node 12/pod 10/ethernet/Speed.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 509 OUT291

Tool Calling - Simple: Live Shell, Classification & APIs (set 7) 3 passed

100.0%

IN14 949 OUT478

✓

SimpleCmdKillTimerBFCL/live_simple_147-95-4 (cmd_controller.execute): remove timer.exe via taskkill.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 869 OUT71

✓

SimpleRecordBalanceAndAtmBFCL/live_simple_111-67-1 (record): 10 queries -> 8 balance + 2 atm; other buckets empty.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 204 OUT308

✓

SimpleReqDeviceConnectorVersionsBFCL/live_simple_132-85-0 (requests.get): device connector versions JSON url, no params.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 876 OUT99

Tool Calling - Simple: Live Shell, Classification & APIs (set 8) 2 passed1 failed

66.7%

IN21 639 OUT764

×

SimpleCmdWhatFileInCBFCL/live_simple_148-95-5 (cmd_controller.execute): 'what is the file in c' => 'dir C:\'. EXPECTED MODEL MISS: the 27B mis-selects the 'record' classifier (dropping the request into its 'outofscope' bucket) instead of cmd_controller_execute. record is a legitimate distractor here (not a looping universal collider), so this stays as a real tool-selection failure — no isolation needed.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 666 OUT405

✓

SimpleRecordWeatherOutOfScopeBFCL/live_simple_112-68-0 (record): single query 'what is the weather like' -> outofscope only.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 007 OUT196

✓

SimpleReqRecommendationsDacBFCL/live_simple_133-86-0 (requests.get): recommendations url, params advisoryId='dac'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 966 OUT163

Tool Calling - Simple: Live Shell, Classification & APIs (set 9) 3 passed

100.0%

IN13 883 OUT285

✓

SimpleCalculateSum133And34BFCL/live_simple_115-71-0 (calculate_sum): number1=133, number2=34.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 611 OUT74

✓

SimpleCmdEchoHiBFCL/live_simple_149-95-6 (cmd_controller.execute): say hi via echo command.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 585 OUT57

✓

SimpleReqInsightsGroupsBFCL/live_simple_134-87-0 (requests.get): insights groups url, params limit=20, sort='desc'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 687 OUT154

Tool Calling - Simple: Math, Physics & Geometry (set 1) 12 passed

100.0%

IN88 670 OUT1 707

✓

SimpleAlgebraQuadraticRootsBFCL/simple_python_3 (algebra.quadratic_roots): a=1, b=-3, c=2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 399 OUT93

✓

SimpleAreaUnderCurveXSquaredBFCL/simple_python_13 (calculate_area_under_curve): function y=x^2, interval [1,3], method omitted/trapezoidal.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 396 OUT191

✓

SimpleCircleCircumferenceInchesBFCL/simple_python_7 (calculate_circumference): radius=4, unit inches/in.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 380 OUT105

✓

SimpleDerivativeCubicPolyBFCL/simple_python_14 (calculate_derivative): function 3x^2+2x-1, x_value omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 387 OUT92

✓

SimpleElectricFieldFromChargeBFCL/simple_python_39 (calculate_electric_field): charge=2, distance=3, permitivity omitted/8.854e-12.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 388 OUT192

✓

SimpleFinalVelocityFromDistanceBFCL/simple_python_30 (kinematics.final_velocity_from_distance): acceleration=4, distance=300, initial_velocity omitted/0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 424 OUT199

✓

SimpleFinalVelocityFromHeightBFCL/simple_python_25 (calculate_final_velocity): height=150, initial_velocity omitted/0, gravity omitted/9.81.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 396 OUT297

✓

SimpleGeometryAreaCircleBFCL/simple_python_8 (geometry.area_circle): radius=10, units omitted or 'meters'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 369 OUT108

✓

SimpleGetPrimeFactors450BFCL/simple_python_17 (get_prime_factors): number=450, formatted omitted or true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 359 OUT72

✓

SimpleMagneticFieldLoopBFCL/simple_python_40 (calculate_magnetic_field): current=5, radius=4, permeability omitted/12.57e10.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 400 OUT203

✓

SimpleMathGcd40And50BFCL/simple_python_19 (math.gcd): num1=40, num2=50.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 383 OUT74

✓

SimpleTriangleAreaWithUnitBFCL/simple_python_0 (calculate_triangle_area): base=10, height=5, unit omitted or 'units'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 389 OUT81

Tool Calling - Simple: Math, Physics & Geometry (set 2) 12 passed

100.0%

IN88 354 OUT2 004

✓

SimpleAverageVelocityBFCL/simple_python_26 (calculate_velocity): distance=50, duration=2, unit omitted or 'km/h'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 363 OUT65

✓

SimpleCalculusDerivativeAtValueBFCL/simple_python_16 (calculus.derivative): function 2x^2, value=1, function_variable omitted or 'x'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 354 OUT194

✓

SimpleElectricFieldStrengthBFCL/simple_python_44 (calculate_electric_field_strength): charge=0.01, distance=4, medium omitted or 'vacuum'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 364 OUT103

✓

SimpleFinalVelocityConstAccelBFCL/simple_python_31 (calculate_final_velocity): initial_velocity=0, acceleration=9.8, time=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 401 OUT190

✓

SimpleGeometryCalculateAreaCircleBFCL/simple_python_9 (geometry.calculate_area_circle): radius=5, unit omitted or 'units'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 338 OUT113

✓

SimpleGeometryCircumferenceBFCL/simple_python_12 (geometry.circumference): radius=3, units omitted or 'cm'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 331 OUT78

✓

SimpleIntegrateCubeSimpsonBFCL/simple_python_15 (integrate): function x^3, start_x=-2, end_x=3, method simpson.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 396 OUT238

✓

SimpleMagneticFieldStrengthWireBFCL/simple_python_43 (calculate_magnetic_field_strength): current=20, distance=10, permeability omitted/12.57e-7.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 371 OUT149

✓

SimpleMathHcfBFCL/simple_python_20 (math.hcf): number1=36, number2=24.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 349 OUT74

✓

SimpleNumberAnalysisPrimeFactorsBFCL/simple_python_18 (number_analysis.prime_factors): number=123456.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 345 OUT381

✓

SimpleRightTriangleAreaCmBFCL/simple_python_10 (calculate_area): base=6, height=10, unit omitted or 'cm'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 380 OUT93

✓

SimpleSolveQuadraticEquation265BFCL/simple_python_4 (solve_quadratic_equation): a=2, b=6, c=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 362 OUT326

Tool Calling - Simple: Math, Physics & Geometry (set 3) 9 passed

100.0%

IN60 246 OUT2 110

✓

SimpleBoilingMeltingPointsBFCL/simple_python_47 (get_boiling_melting_points): substance='water', sea_level=5000.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 688 OUT212

✓

SimpleDisplacementBFCL/simple_python_28 (calculate_displacement): initial_velocity=10, time=5, acceleration=9.8.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 706 OUT232

✓

SimpleElectrostaticPotentialBFCL/simple_python_38 (calculate_electrostatic_potential): charge1=1e-9, charge2=2e-9, distance=0.05, constant omitted/8.99e9.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 720 OUT875

✓

SimpleFinalSpeedFromHeightBFCL/simple_python_32 (calculate_final_speed): initial_velocity=0, height=100, gravity omitted or 9.8.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 676 OUT218

✓

SimpleNumberTheoryGcdBFCL/simple_python_21 (number_theory.gcd): number1=36, number2=48.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 689 OUT81

✓

SimplePrimeFactorizeDictionaryBFCL/simple_python_23 (prime_factorize): number=60, return_type='dictionary'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 680 OUT110

✓

SimpleRouteEstimateTimeStopsBFCL/simple_python_37 (route.estimate_time): SF -> LA, stops {Santa Barbara, Monterey} in either order.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 698 OUT118

✓

SimpleSolveQuadraticAllRootsBFCL/simple_python_5 (solve_quadratic): a=3, b=-11, c=-4, root_type='all' (user asked for all roots).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 709 OUT105

✓

SimpleTriangleAreaNoUnitBFCL/simple_python_11 (calculate_triangle_area): base=10, height=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 680 OUT159

Tool Calling - Simple: Math, Physics & Geometry (set 4) 9 passed

100.0%

IN59 774 OUT1 393

✓

SimpleCalculateDensityBFCL/simple_python_48 (calculate_density): mass=45, volume=15, unit omitted or 'kg/m^3'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 644 OUT98

✓

SimpleElectromagneticForceBFCL/simple_python_41 (electromagnetic_force): charge1=5, charge2=7, distance=3, medium_permittivity omitted/8.854e-12.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 646 OUT228

✓

SimpleFinalSpeedFreeFallTimeBFCL/simple_python_29 (calculate_final_speed): initial_speed omitted/0, time=5, gravity omitted/-9.81.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 631 OUT181

✓

SimpleMathFactorial5BFCL/simple_python_1 (math.factorial): number=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 604 OUT56

✓

SimpleMathGcd12And15BFCL/simple_python_22 (math.gcd): num1=12, num2=15.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 637 OUT73

✓

SimpleShortestDrivingDistanceBFCL/simple_python_36 (get_shortest_driving_distance): origin=New York City, destination=Washington D.C., unit omitted/km.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 629 OUT148

✓

SimpleSolveQuadraticRoots253BFCL/simple_python_6 (solve_quadratic): a=2, b=5, c=3.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 642 OUT350

✓

SimpleThermoEnergyVaporizationBFCL/simple_python_45 (thermo.calculate_energy): mass=100, phase_transition=vaporization, substance omitted/water.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 674 OUT139

✓

SimpleTravelItineraryTokyoBFCL/simple_python_34 (travel_itinerary_generator): destination=Tokyo, days=7, daily_budget=100, exploration_type=nature.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 667 OUT120

Tool Calling - Simple: Math, Physics & Geometry (set 5) 8 passed

100.0%

IN51 582 OUT1 151

✓

SimpleAbsolutePressureBFCL/simple_python_49 (calc_absolute_pressure): atm_pressure=1, gauge_pressure=2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 441 OUT260

✓

SimpleFinalTemperatureMixBFCL/simple_python_46 (calculate_final_temperature): mass1=20, temperature1=30, mass2=15, temperature2=60, shc omitted/4.2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 479 OUT119

✓

SimpleFinalVelocityAccelTimeBFCL/simple_python_27 (final_velocity): initial_velocity=10, acceleration=2, time=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 473 OUT175

✓

SimpleGetDirectionsFastestBFCL/simple_python_33 (get_directions): start=Sydney, end=Melbourne, route_type omitted or 'fastest'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 426 OUT100

✓

SimpleMathGcd12And18BFCL/simple_python_24 (math.gcd): num1=12, num2=18.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 427 OUT77

✓

SimpleMathHypotenuseBFCL/simple_python_2 (math.hypot): x=4, y=5, z omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 437 OUT157

✓

SimpleResonantFrequencyLCBFCL/simple_python_42 (calculate_resonant_frequency): inductance=0.05 (50mH), capacitance=0.0001 (100uF), round_off omitted/2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 464 OUT108

✓

SimpleVeganRestaurantFindBFCL/simple_python_35 (vegan_restaurant.find_nearby): location='New York, NY', operating_hours=23 (11 PM).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 435 OUT155

Tool Calling - Simple: Science, Data & Biology (set 1) 13 passed

100.0%

IN107 062 OUT1 738

✓

SimpleArraySortBFCL/simple_python_87 (array_sort): list=[5,3,4,1,2], order=ascending.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 217 OUT104

✓

SimpleAverageTemperatureBFCL/simple_python_78 (average_temperature): location=Austin, days=3, temp_unit=Celsius.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 201 OUT102

✓

SimpleBacteriaEvolutionRateBFCL/simple_python_74 (calculate_bacteria_evolution_rate): start=5000, dup_freq=1, duration=6, gen_time omitted or 20.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 234 OUT150

✓

SimpleBmiKgCmBFCL/simple_python_84 (calculate_bmi): weight=85, height=180, unit omitted or 'metric'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 220 OUT136

✓

SimpleCreateHistogramBFCL/simple_python_79 (create_histogram): data=[85,90,88,92,86,89,91], bins=5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 277 OUT259

✓

SimpleCreatureFitnessBFCL/simple_python_72 (calculate_fitness): trait_values=[0.8,0.7], trait_contributions=[0.4,0.6].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 280 OUT128

✓

SimpleDbFetchStudentsBFCL/simple_python_89 (db_fetch_records): database=StudentDB, table=students, conditions={department:Science, school:Bluebird High School}, fetch_limit omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 231 OUT105

✓

SimpleEntropyChangeSubstanceBFCL/simple_python_50 (entropy_change.calculate): substance=ice, mass=1, init=0, final=100, pressure omitted or 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 269 OUT167

✓

SimpleGenerateDnaSequenceBFCL/simple_python_71 (generate_DNA_sequence): length=100, preferences=[G,C] in either order.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 220 OUT108

✓

SimpleGpsCoordinateDistanceBFCL/simple_python_83 (calculate_distance): coord1=[33.4484,-112.074], coord2=[34.0522,-118.2437], unit=miles.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 309 OUT179

✓

SimpleImdbMoviesByActorBFCL/simple_python_92 (imdb.find_movies_by_actor): actor='Leonardo DiCaprio', year=2010, category omitted or 'all'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 214 OUT90

✓

SimplePopulationProjectionBFCL/simple_python_73 (population_projections): country=United States/USA, years=20, growth_rate omitted or 1.2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 197 OUT76

✓

SimpleRestaurantFindNearbyBFCL/simple_python_77 (restaurant.find_nearby): location='Los Angeles, CA', dietary_preference=[Vegan].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 193 OUT134

Tool Calling - Simple: Science, Data & Biology (set 2) 13 passed

100.0%

IN106 698 OUT1 747

✓

SimpleBmiMetersBFCL/simple_python_88 (calculate_BMI): weight_kg=70, height_m=1.75.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN8 206 OUT153

✓

SimpleClockAngleBFCL/simple_python_98 (calculate_clock_angle): hours=6, minutes=30, round_to omitted or 2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 190 OUT78

✓

SimpleDatabaseQueryConditionsBFCL/simple_python_96 (database.query): table=user, conditions=[{age,>,25},{job,=,engineer}].1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 249 OUT155

✓

SimpleElephantPopulationBFCL/simple_python_75 (elephant_population_estimate): current=35000, growth_rate=0.015, years=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN8 230 OUT131

✓

SimpleEntropyChangeIsothermalBFCL/simple_python_51 (calculate_entropy_change): initial_temp=300, final_temp=400, heat_capacity=5, isothermal omitted or true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 238 OUT217

✓

SimpleEvolutionPredictionDarwinBFCL/simple_python_76 (prediction.evolution): species=Homo Sapiens, years=50, model=Darwin.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 204 OUT107

✓

SimpleFindRestaurantsThaiBFCL/simple_python_80 (find_restaurants): location=Manhattan variants, food_type=Thai, number=5, dietary_requirements=[vegan].1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 212 OUT233

✓

SimpleGeoDistanceCitiesBFCL/simple_python_85 (geo_distance.calculate): start='Boston, MA', end='Washington, D.C.', units omitted or 'miles'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 200 OUT112

✓

SimpleMathFactorialBFCL/simple_python_97 (math.factorial): number=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN8 153 OUT83

✓

SimplePlotSineWaveBFCL/simple_python_99 (plot_sine_wave): start=0.0, end=6.2832, frequency=5, amplitude omitted or 1, phase_shift omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 201 OUT109

✓

SimpleTheaterMovieReleasesBFCL/simple_python_93 (get_theater_movie_releases): location=LA, timeframe=7, format=IMAX.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 195 OUT144

✓

SimpleTriangleAreaMetersBFCL/simple_python_95 (calc_area_triangle): base=5, height=3.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN8 179 OUT109

✓

SimpleUpdateUserInfoBFCL/simple_python_94 (update_user_info): user_id=43523, update_info={name:John Doe, email:johndoe@email.com}, database omitted or CustomerInfo. NOTE: single-call contract kept intentionally; the model very occasionally emits a spurious second call (args are correct ~3/3 sampled runs), a model-reliability signal rather than a test defect.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 241 OUT116

Tool Calling - Simple: Science, Data & Biology (set 3) 11 passed1 failed

91.7%

IN95 542 OUT1 886

✓

SimpleCalculateAverageBFCL/simple_python_82 (calculate_average): numbers=[12,15,18,20,21,26,30].1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 664 OUT167

✓

SimpleCellDensityBFCL/simple_python_57 (calculate_cell_density): optical_density=0.6, dilution=5, calibration_factor omitted or 1e9.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 636 OUT149

✓

SimpleCellInfoHumanBFCL/simple_python_55 (biology.get_cell_info): cell_type=human, detailed=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 600 OUT358

×

SimpleCellProteinsMembraneBFCL/simple_python_56 (cellbio.get_proteins): cell_compartment=plasma membrane, include_description omitted/true/false.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 633 OUT374

✓

SimpleCityDistanceTrainBFCL/simple_python_86 (city_distance.find_shortest): start=New York, end=Los Angeles, transportation=train, allow_transfer=true.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 644 OUT110

✓

SimpleEmployeeFetchDataBFCL/simple_python_90 (employee.fetch_data): company='ABC Ltd.', employee_id=345, data_field=[Personal Info, Job History].1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 645 OUT95

✓

SimpleFetchDnaSequenceBFCL/simple_python_53 (fetch_DNA_sequence): DNA_id=DNA123, format omitted or 'fasta', upstream omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 603 OUT76

✓

SimpleGetRestaurantSushiBFCL/simple_python_91 (get_restaurant): cuisine=sushi, location=Boston, condition=open on Sundays.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 618 OUT91

✓

SimpleHeatCapacityAirBFCL/simple_python_52 (calc_heat_capacity): temp=298, volume=10, gas omitted or 'air'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 647 OUT108

✓

SimpleMolecularWeightBFCL/simple_python_59 (calculate_molecular_weight): compound=C6H12O6, to_unit=grams/mole or g/mol.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 630 OUT145

✓

SimpleOrganelleFunctionLookupBFCL/simple_python_58 (cell_biology.function_lookup): molecule=ATP synthase, organelle=mitochondria, specific_function=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 619 OUT143

✓

SimpleProteinSequenceBFCL/simple_python_54 (get_protein_sequence): gene=BRCA1, species omitted or 'Homo sapiens'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 603 OUT70

Tool Calling - Simple: Science, Data & Biology (set 4) 12 passed

100.0%

IN94 184 OUT1 778

✓

SimpleAnalyzeDnaSequenceBFCL/simple_python_62 (analyze_dna_sequence): sequence + reference_sequence given, mutation_type omitted or 'substitution'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 904 OUT163

✓

SimpleDiabetesPredictionBFCL/simple_python_61 (diabetes_prediction): weight=150, height=70 (5ft10in), activity_level=lightly active.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 865 OUT113

✓

SimpleFastestRouteAvoidTollsBFCL/simple_python_81 (map_routing.fastest_route): start=San Francisco/SF, end=Los Angeles/LA, avoid_tolls=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 834 OUT99

✓

SimpleForestGrowthForecastBFCL/simple_python_68 (forest_growth_forecast): location=Yellowstone National Park, years=5, include_human_impact=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 841 OUT151

✓

SimpleGeneticSimilarityBFCL/simple_python_63 (genetics.calculate_similarity): species1=Human, species2=Chimp/Chimpanzee, format omitted or 'percentage'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 832 OUT144

✓

SimpleGenotypeFrequencyBFCL/simple_python_64 (calculate_genotype_frequency): allele_frequency=0.3, genotype=AA.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 843 OUT132

✓

SimpleIdentifyBirdBFCL/simple_python_67 (identify_bird): color=green, habitat=forest, size=small.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 817 OUT90

✓

SimpleMutationTypeFindBFCL/simple_python_60 (mutation_type.find): snp_id=rs6034464, species omitted or 'Homo sapiens'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 847 OUT134

✓

SimplePopulationDensityBFCL/simple_python_65 (calculate_density): country=Brazil, year='2022', population=213000000, land_area=8500000.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 890 OUT237

✓

SimplePrecipitationStatsBFCL/simple_python_66 (ecology_data.precipitation_stats): location=Amazon rainforest, time_frame=six_months.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 832 OUT130

✓

SimpleTurtlePopulationBFCL/simple_python_69 (ecology.get_turtle_population): location=Mississippi river, year=2020, species=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 840 OUT227

✓

SimpleVehicleEmissionBFCL/simple_python_70 (calculate_vehicle_emission): vehicle_type=gas, miles_driven=1500, emission_factor omitted or 355.48.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 839 OUT158

Tool Calling - Simple: Sports & Games (set 1) 12 passed

100.0%

IN91 976 OUT1 806

✓

SimpleBattingAverageComputeBFCL/simple_python_302 (calculate_batting_average): hits=180, at_bats=600, decimal_places=3 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 671 OUT99

✓

SimpleBoardGameInfoTicketBFCL/simple_python_328 (boardgame.get_info): Ticket to Ride, {rating, player count} either order, language omitted/English.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 640 OUT257

✓

SimpleFindCardInDeckBFCL/simple_python_335 (find_card_in_deck): rank Queen, suit Hearts, deck omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 633 OUT66

✓

SimpleGameListByYearBFCL/simple_python_342 (game_list.get_games): year 2019, multiplayer true, ESRB Everyone.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 654 OUT134

✓

SimpleGameWinnerByDateBFCL/simple_python_307 (game_result.get_winner): Lakers/Clippers either order, date 2021-01-28, venue omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 652 OUT93

✓

SimpleHighestGameScoreBFCL/simple_python_349 (game_score.highest): Overwatch, PC, region Global or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 645 OUT95

✓

SimpleMatchScheduleBFCL/simple_python_308 (sports.match_schedule): Man United, 5 matches, English Premier League or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 650 OUT98

✓

SimpleNoteDurationBFCL/simple_python_300 (music.calculate_note_duration): 440/880 Hz, tempo omitted or 120.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 652 OUT106

✓

SimplePokerGameWinnerMultiBFCL/simple_python_337 (poker_game_winner): 4 players + per-player hands dict, type omitted/Texas Holdem.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 843 OUT220

✓

SimpleSoccerPlayerStatsBFCL/simple_python_303 (soccer_stat.get_player_stats): Cristiano Ronaldo, 2019-2020, league omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 650 OUT233

✓

SimpleSportRankingPlayerBFCL/simple_python_316 (get_sport_ranking): Tennis, Serena Williams, gender omitted/all/female.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 635 OUT280

✓

SimpleTeamRankRegularBFCL/simple_python_317 (get_team_rank): LA Lakers, NBA, 2021, type regular.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 651 OUT125

Tool Calling - Simple: Sports & Games (set 2) 11 passed

100.0%

IN78 512 OUT2 294

✓

SimpleBoardGameInfoCatanBFCL/simple_python_330 (board_game_info): Catan, {average_review_rating, age_range}.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 128 OUT127

✓

SimpleCricketBattingScoreBFCL/simple_python_306 (average_batting_score): Virat Kohli, 10 matches, T20 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 129 OUT95

✓

SimpleFifaTeamRankingBFCL/simple_python_318 (get_team_ranking): Germany, 2021, gender men or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 136 OUT133

✓

SimpleMatchResultsTwoTeamsBFCL/simple_python_325 (sports.match_results): Chicago Bulls vs Los Angeles Lakers, season omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 114 OUT190

✓

SimplePokerDetermineWinnerTwoBFCL/simple_python_340 (card_games.poker_determine_winner): John vs Mike with 5-card hands (suit symbols).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 262 OUT690

✓

SimpleShuffleAndDrawBFCL/simple_python_336 (cards.shuffle_and_draw): num_cards 3.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 108 OUT61

✓

SimpleSoccerPerformanceBFCL/simple_python_305 (sports_stats.get_performance): Messi, La Liga, 2020-2021, {goals scored, assists made}.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 164 OUT344

✓

SimpleTeamScheduleGamesBFCL/simple_python_327 (sports_team.get_schedule): Man United, 6 games, Premier League, location omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 132 OUT105

✓

SimpleThirdChordBFCL/simple_python_301 (get_third_chord): key C, type major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 106 OUT392

✓

SimpleTopPlayerBySportBFCL/simple_python_323 (sports_ranking.get_top_player): tennis, gender women.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 109 OUT73

✓

SimpleVideoGamePlayerStatsBFCL/simple_python_343 (game_stats.fetch_player_statistics): Zelda, user Sam, platform Switch.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 124 OUT84

Tool Calling - Simple: Sports & Games (set 3) 9 passed

100.0%

IN58 114 OUT1 781

✓

SimpleCardDrawProbabilityBFCL/simple_python_338 (card_game_probability.calculate): total 52, desired 13 (hearts), cards_drawn 1 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 468 OUT129

✓

SimpleCelebrityNetWorthBFCL/simple_python_313 (celebrity_net_worth.get): Messi, currency EUR/euro.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 450 OUT221

✓

SimpleDefenseRankingBFCL/simple_python_315 (get_defense_ranking): season 2021, top 1 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 459 OUT128

✓

SimpleFindAthleteBFCL/simple_python_311 (sports_db.find_athlete): Lebron James, sport Basketball, team omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 444 OUT484

✓

SimpleGameItemStatsBFCL/simple_python_344 (get_game_item_stats): BotW, Guardian Sword+, stat power/power rating.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 487 OUT94

✓

SimpleMonopolyOddsBFCL/simple_python_329 (monopoly_odds_calculator): number 7, dice_number 2, dice_faces 6 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 456 OUT137

✓

SimpleNflPlayerRecordBFCL/simple_python_309 (nfl_data.player_record): Tom Brady, 2020, team omitted or Tampa Bay Buccaneers.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 458 OUT246

✓

SimpleSportsRankingSeasonIntBFCL/simple_python_319 (sports_ranking): Man United, Premier League, season omitted or 2023.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 442 OUT179

✓

SimpleTeamLatestScoreBFCL/simple_python_324 (team_score.get_latest): Los Angeles Lakers, include_opponent true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 450 OUT163

Tool Calling - Simple: Sports & Games (set 4) 7 passed

100.0%

IN41 877 OUT1 433

✓

SimpleCareerStatsBFCL/simple_python_310 (get_career_stats): LeBron James, team omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 958 OUT241

✓

SimpleCelebrityAchievementsBFCL/simple_python_314 (sports_celebrity.get_major_achievements): Messi, sports Football/Soccer or omitted, team all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 976 OUT420

✓

SimpleChessTopPlayersBFCL/simple_python_331 (board_game.chess.get_top_players): New York, min rating 2300, number_of_players 10 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 983 OUT134

✓

SimpleDeckOddsSuitBFCL/simple_python_341 (deck_of_cards.odds): suit hearts, deck_type without_joker or normal.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 975 OUT106

✓

SimpleGameValuationBFCL/simple_python_345 (game_valuation): Super Mario Bros., 1985, condition Like New/New.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 995 OUT176

✓

SimpleTeamPositionDetailedBFCL/simple_python_320 (sports_ranking.get_team_position): Golden State Warriors, 2022-2023, detailed true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 011 OUT218

✓

SimpleTeamScoreWithStatsBFCL/simple_python_326 (get_team_score): Los Angeles Lakers, NBA, include_player_stats any.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 979 OUT138

Tool Calling - Simple: Sports & Games (set 5) 7 passed

100.0%

IN40 540 OUT927

✓

SimpleChessRatingBFCL/simple_python_332 (chess.rating): Magnus Carlsen, variant classical or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 776 OUT71

✓

SimpleCollectablesInSeasonBFCL/simple_python_346 (get_collectables_in_season): Animal Crossing: New Horizons, Spring, item_type all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 799 OUT86

✓

SimpleDetailedWeatherForecastBFCL/simple_python_333 (detailed_weather_forecast): London UK, 3 days, {high/low temp, humidity, precipitation}.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 821 OUT125

✓

SimpleFullHouseProbabilityBFCL/simple_python_339 (poker_probability.full_house): defaults deck_size 52, hand_size 5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 777 OUT122

✓

SimplePlayerStatisticYearBFCL/simple_python_312 (player_statistic): Ronaldo, year 2021, team omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 783 OUT247

✓

SimpleSoccerLastMatchBFCL/simple_python_347 (soccer.get_last_match): Liverpool F.C., include_stats true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 789 OUT188

✓

SimpleSportsRankingSeasonStrBFCL/simple_python_321 (sports_ranking): Barcelona, La Liga, 2021.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 795 OUT88

Tool Calling - Simple: Sports & Games (set 6) 4 passed

100.0%

IN20 218 OUT648

✓

SimpleBlackjackWinnerBFCL/simple_python_334 (blackjack.check_winner): player [A,10], dealer [10,9], ace_value 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 084 OUT317

✓

SimpleCreatePlayerProfileBFCL/simple_python_348 (create_player_profile): StarPlayer, class Mage, starting_level 5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 061 OUT89

✓

SimpleCurrentTeamRankingBFCL/simple_python_322 (sports_ranking.get_current): Liverpool, Premier League, season omitted or 2023-2024.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 027 OUT114

✓

SimplePlayerLastGameStatsBFCL/simple_python_304 (player_stats.getLastGame): LeBron James, Lakers, metrics {Points, Rebounds}.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 046 OUT128

Tool Calling - Simple: Stats, Finance & ML (set 1) 8 passed

100.0%

IN51 793 OUT1 381

✓

SimpleCardProbabilityBFCL/simple_python_112 (calculate_probability): total_outcomes=52, favorable_outcomes=4 (king from deck), round_to omitted or 2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 473 OUT258

✓

SimpleCompoundInterestNBFCL/simple_python_131 (calculate_compound_interest): principal=10000, rate=0.05, time=10, n=4.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 513 OUT177

✓

SimpleInvestmentProfitBFCL/simple_python_134 (investment.predictProfit): investment_amount=5000, annual_return=0.07, years=5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 458 OUT217

✓

SimpleLightTravelTimeBFCL/simple_python_100 (light_travel_time): distance_in_light_years=4, speed_of_light omitted or default.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 421 OUT104

✓

SimpleQuarterlyDividendBFCL/simple_python_128 (finance.calculate_quarterly_dividend_per_share): total_payout=50000000, outstanding_shares=100000000.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 469 OUT154

✓

SimpleRandomForestTrainBFCL/simple_python_109 (random_forest.train): n_estimators=100, max_depth=5, data='my_data'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 447 OUT111

✓

SimpleStockPriceByDateBFCL/simple_python_142 (get_stock_price): company_name Amazon/AMZN, date=2022-03-11, exchange NASDAQ or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 449 OUT110

✓

SimpleTTestStatsBFCL/simple_python_118 (stats.t_test): array_1=[10,15,12,14,11], array_2=[18,16,17,20,22], alpha=0.05.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 563 OUT250

Tool Calling - Simple: Stats, Finance & ML (set 2) 8 passed

100.0%

IN53 952 OUT1 315

✓

SimpleCalculateSpeedBFCL/simple_python_101 (calculate_speed): distance=450, time=20, to_unit=km/h.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 721 OUT141

✓

SimpleCardProbabilitySingleBFCL/simple_python_116 (probabilities.calculate_single): total_outcomes=52, event_outcomes=4 (king), round omitted or 2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 709 OUT140

✓

SimpleCompoundInterestFreqBFCL/simple_python_136 (compound_interest): principal=10000, annual_rate=5.0, compounding_freq=monthly, time_in_years=5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 746 OUT216

✓

SimpleDiscountedCashFlowBFCL/simple_python_129 (calculate_discounted_cash_flow): coupon_payment=100, period=5, discount_rate=0.04, face_value omitted or 1000.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 741 OUT97

✓

SimpleHousePriceBedroomsBFCL/simple_python_110 (predict_house_price): bedrooms=3, bathrooms=2, area=1800, location San Francisco.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 742 OUT131

✓

SimpleStockPriceDaysBFCL/simple_python_143 (get_stock_price): company=AAPL, days=5, exchange NASDAQ or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 702 OUT104

✓

SimpleStockReturnProjectedBFCL/simple_python_137 (calculate_stock_return): investment_amount=5000, annual_growth_rate=0.06, holding_period=5, dividends omitted or false.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 754 OUT223

✓

SimpleTTestIndBFCL/simple_python_119 (hypothesis_testing.ttest_ind): sample1=[22,33,42,12,34], sample2=[23,45,44,14,38], significance_level=0.05.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 837 OUT263

Tool Calling - Simple: Stats, Finance & ML (set 3) 8 passed

100.0%

IN52 242 OUT1 195

✓

SimpleCompoundedInterestAnnuallyBFCL/simple_python_145 (calculate_compounded_interest): principal=5000, interest_rate=0.05, period=10, compounding_frequency omitted or Annually.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 527 OUT132

✓

SimpleDistanceEarthMoonBFCL/simple_python_102 (calculate_distance): body1=Earth, body2=Moon, unit mi/miles/mile.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 483 OUT149

✓

SimpleHeartRatioProbabilityBFCL/simple_python_117 (probability_of_event): success_outcomes=13 (hearts), total_outcomes=52, format_as_ratio=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 515 OUT93

✓

SimpleHousePriceRoomsBFCL/simple_python_125 (predict_house_price): area=2500, rooms=5, year=1990, location San Francisco/SF.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 542 OUT143

✓

SimpleMutualFundReturnBFCL/simple_python_139 (estimate_mutual_fund_return): yearly_yield=5.0, investment_amount=2000, years=3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 543 OUT145

✓

SimpleReturnOnEquityBFCL/simple_python_132 (calculate_return_on_equity): net_income=2000000, shareholder_equity=10000000, dividends_paid=200000.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 585 OUT181

✓

SimpleStockPriceCurrentMultiBFCL/simple_python_149 (get_stock_price): company_names=[Apple, Microsoft] (or tickers AAPL, MSFT).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 465 OUT199

✓

SimpleTwoSampleTTestRunBFCL/simple_python_120 (run_two_sample_ttest): group1=[3,4,5,6,4], group2=[7,8,9,8,7], equal_variance=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 582 OUT153

Tool Calling - Simple: Stats, Finance & ML (set 4) 8 passed

100.0%

IN52 897 OUT1 469

✓

SimpleBinomialProbDistBFCL/simple_python_114 (prob_dist.binomial): trials=10, successes=5, p omitted or 0.5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 578 OUT134

✓

SimpleFutureValueMonthlyBFCL/simple_python_133 (finance.predict_future_value): present_value=5000, annual_interest_rate=0.05, compounding_periods_per_year=12, time_years=3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 620 OUT137

✓

SimpleGeometryAreaTriangleBFCL/simple_python_104 (geometry.area_triangle): base=6, height=10, unit omitted or 'square meters'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 561 OUT129

✓

SimpleMathPowerBFCL/simple_python_105 (math.power): base=3, exponent=4, mod omitted or 1.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 554 OUT71

✓

SimpleNPVCashFlowsBFCL/simple_python_127 (calculate_NPV): cash_flows=[200,300,400,500], discount_rate=0.1, initial_investment=2000.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 669 OUT281

✓

SimpleReturnOnInvestmentBFCL/simple_python_135 (calculate_return_on_investment): purchase_price=20, sale_price=25, dividend=2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 597 OUT247

✓

SimpleStockPriceDataTypeBFCL/simple_python_146 (stock_price): company Amazon/AMZN, days=3, data_type omitted or Close.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 555 OUT69

✓

SimpleTwoSampleTTestFloatBFCL/simple_python_123 (hypothesis_testing.two_sample_t_test): group1=[12.4,15.6,11.2,18.9], group2=[10.5,9.8,15.2,13.8], alpha omitted or 0.05.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 763 OUT401

Tool Calling - Simple: Stats, Finance & ML (set 5) 8 passed

100.0%

IN53 420 OUT1 239

✓

SimpleAreaUnderCurvePolyBFCL/simple_python_103 (mathematics.calculate_area_under_curve): polynomial=[3.0,2.0,-4.0], limits=[-1.0,2.0].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 669 OUT127

✓

SimpleBinomialCalcBFCL/simple_python_115 (calculate_binomial_probability): number_of_trials=8, number_of_successes=5, probability_of_success omitted or 0.5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 648 OUT237

✓

SimpleCalculateBmiBFCL/simple_python_107 (calculate_bmi): weight=70, height=175, system omitted or metric.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 647 OUT102

✓

SimpleFutureValueInvestmentBFCL/simple_python_148 (finance.calculate_future_value): initial_investment=20000, rate_of_return=0.08, years=5, contribution omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 687 OUT137

✓

SimpleLinearRegressionBFCL/simple_python_108 (run_linear_regression): predictors=[Age,Income,Education], target=Purchase_Amount, standardize=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 673 OUT122

✓

SimpleNPVInvestmentBFCL/simple_python_130 (finance_calculator.npv): cash_flows=[-50000,10000,15000,20000,25000,30000], discount_rate=0.08, years omitted or empty.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 771 OUT232

✓

SimpleStockPricesDurationBFCL/simple_python_147 (get_stock_prices): companies=[Microsoft,Google], duration='2 weeks'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 625 OUT76

✓

SimpleTTestDatasetsBFCL/simple_python_124 (t_test): dataset_A=[12,24,36], dataset_B=[15,30,45], alpha omitted or 0.05.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 700 OUT206

Tool Calling - Simple: Stats, Finance & ML (set 6) 10 passed

100.0%

IN71 511 OUT1 405

✓

SimpleBinomialCoin100BFCL/simple_python_121 (calc_binomial_prob): num_trials=100, num_success=60, prob_success=0.5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 157 OUT138

✓

SimpleCagrBFCL/simple_python_140 (calculate_cagr): initial_value=2000, final_value=3000, period_in_years=4.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 176 OUT155

✓

SimpleChiSquaredTestBFCL/simple_python_122 (chi_squared_test): table=[[10,20],[30,40]], alpha omitted or 0.05.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 158 OUT234

✓

SimpleDiceRollBFCL/simple_python_113 (probability.dice_roll): desired_number=6, number_of_rolls=2, die_sides omitted or 6.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 134 OUT187

✓

SimpleMarketPerformanceBFCL/simple_python_144 (market_performance.get_data): indexes=[S&P 500, Dow Jones], days=5, detailed omitted or true/false.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 141 OUT102

✓

SimpleMetalPriceBFCL/simple_python_141 (get_metal_price): metal=Gold, measure=ounce.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 100 OUT80

✓

SimpleNormalVariateBFCL/simple_python_111 (random.normalvariate): mu=0, sigma=1.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 121 OUT106

✓

SimplePortfolioFutureValueBFCL/simple_python_138 (portfolio_future_value): stock=X, invested_amount=5000, expected_annual_return=0.05, years=7.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 179 OUT167

✓

SimpleRandomForestClassifierBFCL/simple_python_106 (train_random_forest_classifier): dataset=your_dataset_name, max_depth=5, n_estimators=100.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 160 OUT105

✓

SimpleRSquaredBFCL/simple_python_126 (linear_regression.get_r_squared): dataset_path=C:/data/cars.csv, independent_variables=[engine_size,fuel_economy], dependent_variable=car_price.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 185 OUT131

Tool Calling - Tool Selection: Area (Triangle/Circle) 1 passed

100.0%

IN4 539 OUT147

✓

SelectTriangleAreaHeronToolBFCL/multiple_1 (math.triangle_area_heron): triangle area from three side lengths (Heron).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 539 OUT147

Tool Calling - Tool Selection: Art & Music 7 passed2 failed

77.8%

IN111 720 OUT2 545

✓

SelectAnalyzeStructureToolBFCL/multiple_162 (analyze_structure): dynamic structural analysis of specific floors.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN11 777 OUT137

✓

SelectConcertSearchToolBFCL/multiple_166 (concert.search): concert search by genre/location/date/price.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 734 OUT184

×

SelectGuitarPriceToolBFCL/multiple_165 (guitar_price.find): guitar price by model/condition/location (not instrument_price.get).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: args.model.ToLower() == "gibson les paul"

Failed

IN11 734 OUT137

✓

SelectInstrumentPriceToolBFCL/multiple_164 (instrument_price.get): instrument price by brand/model/finish (not guitar_price.find).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN11 742 OUT131

✓

SelectMelodyGeneratorToolBFCL/multiple_167 (music_generator.generate_melody): melody generation with key/note/length/tempo. Tightened per methodology: prompt explicitly gives "120 beats per minute", so tempo is required exactly 120 (ground truth also accepts omission, which equals the schema default).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 771 OUT187

×

SelectMetropolitanArtworksToolBFCL/multiple_163 (metropolitan_museum.get_top_artworks): top artworks sorted by popularity.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN17 802 OUT234

✓

SelectMusicalScaleToolBFCL/multiple_169 (musical_scale): musical scale for a key.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 710 OUT94

✓

SelectSculptureInfoToolBFCL/multiple_160 (get_sculpture_info): sculpture info by artist with detail.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 721 OUT212

✓

SelectSongLyricsToolBFCL/multiple_168 (get_song_lyrics): lyrics by song title/artist.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 729 OUT1 229

Tool Calling - Tool Selection: Calculators & Directories 7 passed1 failed

87.5%

IN77 014 OUT1 147

✓

SelectCalculateGcdToolBFCL/multiple_35 (calculate_gcd): greatest common divisor, not lcm.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 595 OUT79

✓

SelectCurrencyGetRateToolBFCL/multiple_17 (currency_conversion.get_rate): rate at a specific date, not convert/latest-rate.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 614 OUT123

✓

SelectGeodistanceToolBFCL/multiple_65 (geodistance.find): distance between cities, not timezone difference/flights.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 584 OUT113

✓

SelectLibrarySearchBookToolBFCL/multiple_38 (library.search_book): search for a book, not reserve one.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 596 OUT179

×

SelectLinearRegressionToolBFCL/multiple_53 (linear_regression): linear regression, not random forest.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 812 OUT355

✓

SelectMagneticFieldCalculateToolBFCL/multiple_41 (magnetic_field.calculate): magnetic field via Ampere's law, not electric field/force.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 627 OUT111

✓

SelectSolveQuadraticEquationToolBFCL/multiple_28 (solve.quadratic_equation): solve a quadratic, not rgb/string-reverse.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 603 OUT102

✓

SelectStockForecastToolBFCL/multiple_55 (stock_forecast): stock price prediction, not weather forecast.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 583 OUT85

Tool Calling - Tool Selection: Finance & Legal 9 passed1 failed

90.0%

IN131 697 OUT1 446

✓

SelectCagrCalculationToolBFCL/multiple_130 (calculate_cagr): compound annual growth rate from initial/final/years, not other finance tools.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 176 OUT111

✓

SelectCaseInfoToolBFCL/multiple_135 (get_case_info): retrieve case info (victim) by docket/court.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 152 OUT107

×

SelectCrimeRateToolBFCL/multiple_136 (get_crime_rate): crime rate by city/state/type/year.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.type.ToLower() == "violent"

Failed

IN13 144 OUT105

✓

SelectCrimeRecordToolBFCL/multiple_134 (crime_record.get_record): detailed crime record by case number/county.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 145 OUT108

✓

SelectFutureValueToolBFCL/multiple_132 (finance.calculate_future_value): future value from rate/initial/years, not compound_interest.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 179 OUT130

✓

SelectLawsuitDetailsToolBFCL/multiple_139 (lawsuit_details.find): patent lawsuit details by company/year/type.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 143 OUT104

✓

SelectLawsuitSearchToolBFCL/multiple_137 (lawsuit_search): lawsuit search by company/date/location/status. Tightened per methodology: prompt explicitly asks for "ongoing" lawsuits, so status is required exactly "ongoing" (ground truth also accepts omission, but an explicitly-stated value must be passed).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 161 OUT120

✓

SelectLegalCaseFetchToolBFCL/multiple_138 (legal_case.fetch): fetch detailed legal case by id.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 292 OUT306

✓

SelectMarketPerformanceToolBFCL/multiple_131 (market_performance.get_data): index performance over N days.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 141 OUT197

✓

SelectMutualFundBalanceToolBFCL/multiple_133 (calculate_mutual_fund_balance): mutual fund balance from investment/yield/years.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 164 OUT158

Tool Calling - Tool Selection: Finance & Markets 8 passed

100.0%

IN82 482 OUT1 034

✓

SelectAvgClosingPriceToolBFCL/multiple_56 (avg_closing_price): average closing price, not volume/total revenue.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 279 OUT74

✓

SelectCompoundInterestCalcToolBFCL/multiple_49 (calc_Compound_Interest): compound interest, not future value/simple interest.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 328 OUT118

✓

SelectCurrencyConversionToolBFCL/multiple_52 (currency_conversion): currency conversion, not unit conversion.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 289 OUT153

✓

SelectDividendDataToolBFCL/multiple_54 (corporate_finance.dividend_data): dividend history, not generic stock market data.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 280 OUT91

✓

SelectHousePriceForecastToolBFCL/multiple_50 (house_price_forecast): house price forecast, not weather/stock forecast.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 273 OUT97

✓

SelectPropertyDepreciationToolBFCL/multiple_14 (finance.property_depreciation): depreciation, not loan/inflation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 328 OUT157

✓

SelectRevenueForecastToolBFCL/multiple_13 (corporate_finance.revenue_forecast): revenue forecast, not product price.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 320 OUT142

✓

SelectYoyGrowthRateToolBFCL/multiple_12 (corporate_finance.calculate_YOY_growth_rate): YOY growth, not ROE/ROA.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 385 OUT202

Tool Calling - Tool Selection: Food & Travel 7 passed

100.0%

IN64 581 OUT1 303

✓

SelectCurrencyConvertToolBFCL/multiple_95 (currency_exchange.convert): currency conversion, not unit_conversion. live_conversion is required true: prompt says "using current exchange rate" (ground truth = [true]).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 227 OUT175

✓

SelectDrivingDistanceToolBFCL/multiple_104 (get_shortest_driving_distance): driving distance, not instrument price / biology / scientist lookup.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 216 OUT111

✓

SelectHotelBookToolBFCL/multiple_93 (hotel.book): book a hotel room, not car.rental.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 245 OUT110

✓

SelectRecipeFindToolBFCL/multiple_91 (recipe.find): recipe by main ingredient + limit, not restaurant.find.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 214 OUT322

✓

SelectRecipeSearchToolBFCL/multiple_89 (recipe_search): find a recipe from ingredients, not restaurant_search/ingredient_replace.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 231 OUT154

✓

SelectRestaurantForGroupToolBFCL/multiple_90 (restaurant.find_group): restaurant for a group, not events.find_event.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 224 OUT104

✓

SelectWalmartVeganToolBFCL/multiple_92 (walmart.vegan_products): the named store is Walmart, not Safeway/Whole Foods.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 224 OUT327

Tool Calling - Tool Selection: Games, Food & Utilities 10 passed

100.0%

IN138 024 OUT1 912

✓

SelectBestGroceryStoreToolBFCL/multiple_188 (grocery_store.find_best): best grocery store by rating/products.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 828 OUT118

✓

SelectClosestRestaurantToolBFCL/multiple_185 (restaurant_search.find_closest): closest restaurant by cuisine/amenities.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 795 OUT83

✓

SelectFindRecipeToolBFCL/multiple_186 (find_recipe): recipe by restrictions/type/time.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 806 OUT329

✓

SelectGamePlayerStatsToolBFCL/multiple_180 (game_stats.fetch_player_statistics): player stats by game/username/platform.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 798 OUT92

✓

SelectMultiplayerGameFinderToolBFCL/multiple_182 (multiplayer_game_finder): multiplayer game by platform/rating.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 802 OUT242

✓

SelectRecipeCaloriesToolBFCL/multiple_183 (recipe_info.get_calories): calories of a recipe from a website.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 793 OUT93

✓

SelectRecipeSearchToolBFCL/multiple_184 (recipe_search): recipe by restriction/ingredients/servings.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 803 OUT456

✓

SelectSoccerLastMatchToolBFCL/multiple_181 (soccer.get_last_match): last match with stats for a club.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 797 OUT257

✓

SelectTimezoneConvertToolBFCL/multiple_189 (timezone.convert): convert a time between timezones.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 799 OUT154

✓

SelectWholeFoodsPriceToolBFCL/multiple_187 (whole_foods.check_price): price of items at a Whole Foods location.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 803 OUT88

Tool Calling - Tool Selection: Geometry, Finance & Services 7 passed

100.0%

IN59 011 OUT845

✓

SelectAirQualityForecastToolBFCL/multiple_63 (air_quality_forecast): air quality, not weather/news.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 413 OUT87

✓

SelectFinancialCompoundInterestToolBFCL/multiple_57 (financial.compound_interest): compound interest, not simple interest.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 476 OUT159

✓

SelectGeometryRectangleToolBFCL/multiple_31 (geometry_rectangle.calculate): rectangle area+perimeter, not square/circle.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 429 OUT76

✓

SelectLawyerFinderToolBFCL/multiple_59 (lawyer_finder): find a lawyer by specialization, not rent a car.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 416 OUT107

✓

SelectLibrarySearchBooksToolBFCL/multiple_68 (library.search_books): search a specific library, not Google Books/Open Library.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 415 OUT79

✓

SelectMapsShortestPathToolBFCL/multiple_27 (maps.shortest_path): shortest path with transport mode, not route-time estimation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 437 OUT204

✓

SelectTrafficEstimateToolBFCL/multiple_66 (traffic_estimate): traffic for a time period, not raw distance/weather.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 425 OUT133

Tool Calling - Tool Selection: History & Knowledge 8 passed2 failed

80.0%

IN143 057 OUT2 403

✓

SelectDiscovererToolBFCL/multiple_155 (get_discoverer): discoverer of a discovery, with detail.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 580 OUT297

✓

SelectEarliestReferenceToolBFCL/multiple_157 (get_earliest_reference): earliest reference of a person from a source.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 592 OUT356

✓

SelectEventDateToolBFCL/multiple_153 (get_event_date): date of a historical event.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 577 OUT70

×

SelectHistoricalContribToolBFCL/multiple_156 (historical_contrib.get_contrib): scientist's contribution on a date.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.category == null) || (args.category?.ToLower() == "all") Values: args.category = "Physics"

Failed

IN13 620 OUT164

✓

SelectHistoryKeyEventsToolBFCL/multiple_152 (history.get_key_events): key war events by country/years.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 625 OUT386

✓

SelectMostFollowedToolBFCL/multiple_151 (social_media_analytics.most_followed): most-followed by topic/sub-topics.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 614 OUT102

✓

SelectNeuronalActivityToolBFCL/multiple_150 (calculate_neuronal_activity): neuronal activity from synaptic rate/weight/decay.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 636 OUT124

✓

SelectPaintNeededToolBFCL/multiple_159 (calculate_paint_needed): paint quantity for a wall.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 643 OUT261

×

SelectPapalBiographyToolBFCL/multiple_158 (religious_history.get_papal_biography): papal biography with contributions.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN20 569 OUT562

✓

SelectUsPresidentToolBFCL/multiple_154 (US_president.in_year): president in a given year. Tightened per methodology: prompt explicitly asks for the "full name", so full_name is required true (ground truth also accepts omission, but an explicitly-requested option must be passed).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 601 OUT81

Tool Calling - Tool Selection: History, Demographics & Art 8 passed2 skipped

100.0%

IN86 017 OUT1 130

✓

SelectArtworkPriceToolBFCL/multiple_74 (art_auction.fetch_artwork_price): artwork price, not library book search. Tightened per methodology: prompt explicitly asks "on all auction platforms", so require platform == "all" (ground truth also accepts omission, but an explicitly-given value must pass).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 768 OUT139

✓

SelectArtworkSearchToolBFCL/multiple_77 (artwork_search.find): artwork search, not tourist_attraction/park search.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 751 OUT263

–

SelectCustomSculptureToolBFCL/multiple_76 (sculpture.create_custom): custom sculpture (material), not painting (color).0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Skipped: Ambiguous action-vs-question item (item flaw, not model behavior): the prompt mixes an order ('I want to order a custom bronze sculpture of a horse') with a question ('What material options are available?'), so a competent model may either call sculpture_create_custom or answer the question in text - not deterministic. Kept for traceability.

Skipped

IN0 OUT0

✓

SelectEuropeanMonarchsToolBFCL/multiple_70 (european_history.get_monarchs): monarch lookup, not events/culture.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 752 OUT76

✓

SelectExhibitionInfoToolBFCL/multiple_79 (exhibition_info): museum exhibitions, not restaurant_info.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 758 OUT156

–

SelectMuseumInfoToolBFCL/multiple_78 (museum_info): museum info with a feature list, not generic tourist_spot_info.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Ambiguous single-call item (item flaw, not model behavior): a dedicated exhibition_info tool coexists with museum_info, so the 'exhibitions' part can reasonably route either to museum_info's features or to a separate exhibition_info call; a competent model may make one or two calls - not deterministic. Kept for traceability.

Skipped

IN0 OUT0

✓

SelectPaintColorTrendsToolBFCL/multiple_75 (paint_color.trends): paint color trend, not weather/house-price trends.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 736 OUT179

✓

SelectPopulationByCategoryToolBFCL/multiple_71 (get_population): population by category, not bureau statistics/demographics.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 760 OUT130

✓

SelectReligionOriginToolBFCL/multiple_73 (religion.get_origin): founder/origin, not core beliefs.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 741 OUT104

✓

SelectStatePopulationToolBFCL/multiple_72 (us_history.population_by_state_year): state population, not GDP.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 751 OUT83

Tool Calling - Tool Selection: Isolated Collisions: Exhibition, Hotel & Currency 3 passed

100.0%

IN21 049 OUT345

✓

SelectBookLuxuryHotelToolBFCL/multiple_191 (book_hotel): luxury room with view, stay_duration variant.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 065 OUT153

✓

SelectCurrencyConversionToolBFCL/multiple_192 (currency_conversion.convert): convert an amount between currencies.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 981 OUT82

✓

SelectExhibitionToolBFCL/multiple_161 (find_exhibition): top-rated sculpture exhibition by location/art_form.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 003 OUT110

Tool Calling - Tool Selection: Knowledge & Media 10 passed

100.0%

IN117 743 OUT1 690

✓

SelectBattleDetailsToolBFCL/multiple_18 (european_history.battle_details): battle details, not war/leader info.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 751 OUT205

✓

SelectDatabaseModifyColumnsToolBFCL/multiple_10 (database.modify_columns): delete columns, not create a backup.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 808 OUT130

✓

SelectFiveFactorModelToolBFCL/multiple_69 (five_factor_model.analyse): Big Five personality, not MBTI.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 822 OUT158

✓

SelectGameRewardsToolBFCL/multiple_26 (game_rewards.get): rewards, not scores/missions list.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 774 OUT185

✓

SelectGenerateSoundWaveToolBFCL/multiple_21 (generate_sound_wave): generate a sound, not play a file.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 775 OUT193

✓

SelectMostPointsSingleGameToolBFCL/multiple_22 (sports_data.basketball.most_points_single_game): single-game record, not season/career.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 751 OUT113

✓

SelectPlayerStatsToolBFCL/multiple_23 (basketball.player_stats.get): player stats, not game/team stats.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 773 OUT207

✓

SelectReligionSchismsToolBFCL/multiple_19 (religion_history.get_schisms): schisms, not councils/reformations.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 744 OUT324

✓

SelectSculpturePriceToolBFCL/multiple_20 (sculpture_price.calculate): commission price, not sculptor info/availability.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 757 OUT78

✓

SelectVideoGameStorePriceToolBFCL/multiple_25 (video_games.store_price): store price, not store currency/on-sale.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 788 OUT97

Tool Calling - Tool Selection: Legal, Environment & Services 10 passed

100.0%

IN121 658 OUT1 452

✓

SelectAirQualityToolBFCL/multiple_144 (air_quality): AQI by location/date.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 155 OUT84

✓

SelectEmissionsToolBFCL/multiple_145 (calculate_emissions): annual CO2 from distance/fuel/efficiency.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 191 OUT166

✓

SelectHumidityForecastToolBFCL/multiple_142 (weather.humidity_forecast): humidity forecast by location/days.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 142 OUT130

✓

SelectLawsuitCheckCaseToolBFCL/multiple_141 (lawsuit.check_case): verify a case's closed status by id.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 161 OUT74

✓

SelectLawsuitsSearchToolBFCL/multiple_140 (lawsuits_search): lawsuits by company/location/year.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 156 OUT109

✓

SelectMapDirectionsToolBFCL/multiple_147 (map_service.get_directions): directions avoiding features.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN12 157 OUT103

✓

SelectRestaurantFindNearbyToolBFCL/multiple_146 (restaurant.find_nearby): nearby restaurants by cuisine/distance.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 152 OUT201

✓

SelectSentimentAnalysisToolBFCL/multiple_149 (sentiment_analysis): sentiment of a review text.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 162 OUT88

✓

SelectSlopeGradientToolBFCL/multiple_143 (calculate_slope_gradient): slope gradient between two coordinates. Tightened per methodology: prompt explicitly asks for the gradient "in degree", so unit is required exactly "degree" (ground truth also accepts omission, which equals the schema default).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN12 251 OUT301

✓

SelectStockInfoToolBFCL/multiple_148 (get_stock_info): detailed stock information by company.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 131 OUT196

Tool Calling - Tool Selection: Local Services & Weather 9 passed

100.0%

IN86 800 OUT1 278

✓

SelectHumidityTemperatureForecastToolBFCL/multiple_60 (humidity_temperature_forecast): humidity/temperature, not events/movies.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 636 OUT125

✓

SelectLandscapeArchitectToolBFCL/multiple_61 (landscape_architect.find_specialty): landscape architect, not home renovation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 651 OUT114

✓

SelectLawyerSearchToolBFCL/multiple_58 (lawyer.search): find a lawyer, not a doctor.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 622 OUT105

✓

SelectLibraryFindNearbyToolBFCL/multiple_48 (library.find_nearby): find a library, not a store.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 647 OUT113

✓

SelectNatureParkToolBFCL/multiple_62 (nature_park.find_nearby): find a nature park, not a restaurant.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 649 OUT299

✓

SelectRideHailingToolBFCL/multiple_39 (ride_hailing.get_rides): find a ride, not a grocery delivery.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 645 OUT97

✓

SelectRoutePlannerToolBFCL/multiple_24 (route_planner.calculate_route): plan a route, not chess-club lookup.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 661 OUT158

✓

SelectTranslateToolBFCL/multiple_67 (translate): translate text, not sentiment/word-count.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 638 OUT88

✓

SelectUvIndexToolBFCL/multiple_64 (uv_index.get_future): UV index, not rainfall/snowfall.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 651 OUT179

Tool Calling - Tool Selection: Mathematics 8 passed

100.0%

IN75 301 OUT1 005

✓

SelectCalculateIntegralToolBFCL/multiple_33 (calculate_integral): definite integral, not derivative.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 417 OUT151

✓

SelectCosineSimilarityToolBFCL/multiple_47 (cosine_similarity.calculate): cosine similarity, not correlation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 436 OUT104

✓

SelectDiceRollProbabilityToolBFCL/multiple_51 (dice_roll_probability): dice sum probability, not coin/card.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 411 OUT154

✓

SelectFunctionsIntersectToolBFCL/multiple_29 (functions.intersect): intersection of two functions, not zeros of one.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 413 OUT267

✓

SelectMathLcmToolBFCL/multiple_34 (math.lcm): least common multiple, not gcd/sqrt.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 403 OUT78

✓

SelectQuadraticRootsToolBFCL/multiple_11 (math_roots.quadratic): quadratic roots from coefficients, not cubic/polynomial.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 415 OUT97

✓

SelectRectangleAreaToolBFCL/multiple_30 (rectangle.area): rectangle area, not circle/triangle area.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 401 OUT76

✓

SelectSortListToolBFCL/multiple_46 (sort_list): sort a list, not filter/sum.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 405 OUT78

Tool Calling - Tool Selection: Mathematics & Geometry 7 passed

100.0%

IN70 351 OUT913

✓

SelectCircleAreaToolBFCL/multiple_97 (geometry.area_circle): circle area, not plot_sine_wave.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 030 OUT85

✓

SelectCircumferenceToolBFCL/multiple_98 (geometry.circumference): circle circumference, not the reference/time/melody tools.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 021 OUT76

✓

SelectDatabaseQueryToolBFCL/multiple_119 (database.query): structured DB query with conditions, not the score/ROI/probability distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN10 112 OUT159

✓

SelectDerivativeToolBFCL/multiple_99 (calculus.derivative): derivative at a point, not get_personality_traits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 054 OUT183

✓

SelectHcfToolBFCL/multiple_100 (math.hcf): highest common factor, not the scale-progression distractor.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 040 OUT78

✓

SelectSolveQuadraticToolBFCL/multiple_96 (solve_quadratic_equation): solve quadratic, not stock/portfolio tools.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 053 OUT221

✓

SelectTriangleAreaToolBFCL/multiple_121 (geometry.area_triangle): triangle area, not map directions / invention lookup.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 041 OUT111

Tool Calling - Tool Selection: Mixed Domains 7 passed1 skipped

100.0%

IN68 858 OUT1 118

✓

SelectAverageGradeToolBFCL/multiple_9 (calculate_average): selects average, not standard_deviation/highest_grade.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 857 OUT146

✓

SelectCapacitanceToolBFCL/multiple_6 (capacitance_calculator.calculate): selects capacitance, not resistance/magnetic_field. Tightened per methodology: prompt explicitly states "dielectric constant K is 1.0", so K is required exactly 1.0 (ground truth also accepts omission, but an explicitly-given value must be passed).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 859 OUT265

✓

SelectCapitalToolBFCL/multiple_2 (country_info.capital): selects capital lookup, not largest_city/population.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 758 OUT54

✓

SelectDisplacementToolBFCL/multiple_4 (kinematics.calculate_displacement): selects displacement, not final_speed.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 851 OUT211

✓

SelectEuclideanDistanceToolBFCL/multiple_3 (EuclideanDistance.calculate): selects distance, not angleToXAxis.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 803 OUT93

✓

SelectRealEstateSearchToolBFCL/multiple_8 (realestate.find_properties): property search with budget, not property_valuation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 862 OUT186

✓

SelectWeatherByCoordinatesDateToolBFCL/multiple_5 (weather.get_by_coordinates_date): historical-by-coordinates, not by-city/forecast.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 868 OUT163

–

SelectWildlifeGrowthToolBFCL/multiple_7 (wildlife_population.assess_growth): population growth, not ecological_impact.0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Skipped: Ambiguous single-call item (item flaw, not model behavior): the prompt asks for BOTH population growth AND impact on woodland, and two distinct tools genuinely fit (wildlife_population_assess_growth + ecological_impact_analyze); a competent human would reasonably make two calls, so a single-call ground truth is unfair. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Tool Selection: Music, Sports & Games 7 passed2 failed

77.8%

IN95 457 OUT2 300

✓

SelectBoardGameRecommendToolBFCL/multiple_86 (BoardGameGeek.recommend): BoardGameGeek store, not AmazonGameStore.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 023 OUT260

✓

SelectConcertTicketToolBFCL/multiple_81 (concert.book_ticket): concert ticket, not festival ticket.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 046 OUT99

✓

SelectGameUpdateToolBFCL/multiple_87 (games.update.find): game update/patch, not reviews/price.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 034 OUT121

✓

SelectMusicGenerateToolBFCL/multiple_82 (music.generate): musical piece from key/tempo, not raw audio.generate.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 012 OUT72

×

SelectMusicShopToolBFCL/multiple_80 (music_shop.find_nearby): music shop with lessons, not gym.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.services.Count == 1 && args.services[0].ToLower() == "violin lessons" Values: args.services.Count = 1

Failed

IN10 021 OUT232

✓

SelectPlayerCountToolBFCL/multiple_88 (video_games.get_player_count): active player count, not sales.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 031 OUT172

✓

SelectPlayerGoalsToolBFCL/multiple_83 (player_stats.get_all_time_goals): per-player goals, not team/league top scorer.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 012 OUT84

×

SelectSoccerScoresToolBFCL/multiple_85 (soccer_scores.get_scores): soccer scores, not basketball.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN15 246 OUT258

✓

SelectTopGoalScorersToolBFCL/multiple_84 (getTopGoalScorers): top goal scorers, not top assists.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 032 OUT1 002

Tool Calling - Tool Selection: Physical Sciences 10 passed

100.0%

IN113 114 OUT1 543

✓

SelectBiologicalEnergyToolBFCL/multiple_43 (biological.calc_energy): energy from moles of substance, not biomass/work.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 295 OUT127

✓

SelectBiotSavartLawToolBFCL/multiple_40 (electromagnetism.biot_savart_law): magnetic field via Biot-Savart, not Ampere's law.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 298 OUT131

✓

SelectConeVolumeToolBFCL/multiple_32 (geometry.calculate_cone_volume): cone volume, not cone mass.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 278 OUT150

✓

SelectFinalTemperatureToolBFCL/multiple_42 (calculate_final_temperature): mixing two gas samples, not mass calculation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 339 OUT117

✓

SelectFinalVelocityToolBFCL/multiple_37 (kinematics.final_velocity): final velocity from initial velocity + time, not wave velocity/distance.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 288 OUT183

✓

SelectGeologyEraToolBFCL/multiple_45 (geology.get_era): geological era date, not historical event date.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 271 OUT122

✓

SelectPopulationGeneticsNeToolBFCL/multiple_16 (population_genetics.calculate_ne): effective population size, not range-shift/carrying-capacity.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 339 OUT222

✓

SelectSolarFarmPotentialToolBFCL/multiple_15 (solarFarm.potential): solar (not wind) energy potential from coordinates + panel area.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 394 OUT159

✓

SelectSpeedFromRestToolBFCL/multiple_36 (kinematics.calculate_speed_from_rest): speed from rest over a distance, not acceleration.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 323 OUT220

✓

SelectWeightInSpaceToolBFCL/multiple_44 (calculate.weight_in_space): weight on a planet, not unit/currency conversion.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 289 OUT112

Tool Calling - Tool Selection: Physics & Biology 12 passed1 failed

92.3%

IN202 769 OUT1 764

×

SelectCellProteinsToolBFCL/multiple_109 (cellbio.get_proteins): proteins by cell compartment, not find_exhibition / mutation lookup.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN22 670 OUT365

✓

SelectDensityToolBFCL/multiple_107 (calculate_density): mass/volume density, not mix_paint_color.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 001 OUT66

✓

SelectDisplacementToolBFCL/multiple_102 (calculate_displacement): kinematic displacement, not poker/musical distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 019 OUT89

✓

SelectElectricFieldToolBFCL/multiple_106 (calculate_electric_field_strength): electric field, not paint/cooking/group distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN14 997 OUT104

✓

SelectEvolutionPredictionToolBFCL/multiple_114 (prediction.evolution): evolutionary-rate prediction, not modify_painting / calculate_probability.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 009 OUT113

✓

SelectFinalSpeedToolBFCL/multiple_103 (calculate_final_speed): free-fall final speed, not chess/court/event distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN14 989 OUT83

✓

SelectFitnessToolBFCL/multiple_113 (calculate_fitness): evolutionary fitness from trait values/contributions, not chess/walmart/lawyer distractors. Percentages map to decimal contributions [0.4, 0.6].1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN15 069 OUT153

✓

SelectForestGrowthToolBFCL/multiple_112 (forest_growth_forecast): forest growth forecast, not get_scientist_for_discovery. include_human_impact is required true: prompt says "including human impact" (ground truth = [true]).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 002 OUT111

✓

SelectGenotypeFrequencyToolBFCL/multiple_111 (calculate_genotype_frequency): genotype frequency, not factorial / find_flute.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 004 OUT130

✓

SelectHeatCapacityToolBFCL/multiple_108 (calc_heat_capacity): gas heat capacity, not discounted_cash_flow.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 011 OUT100

✓

SelectLightTravelTimeToolBFCL/multiple_120 (light_travel_time): light travel time, not grocery/hotel/exchange distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN14 987 OUT99

✓

SelectMagneticFieldToolBFCL/multiple_105 (calculate_magnetic_field): magnetic field of a loop, not concert/lawsuit distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 003 OUT200

✓

SelectMutationTypeToolBFCL/multiple_110 (mutation_type.find): SNP mutation type, not get_collectables_in_season.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 008 OUT151

Tool Calling - Tool Selection: Sports & Card Games 9 passed1 failed

90.0%

IN141 145 OUT1 828

✓

SelectChessTopPlayersToolBFCL/multiple_177 (board_game.chess.get_top_players): top chess players by location/rating.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 427 OUT146

✓

SelectDefenseRankingToolBFCL/multiple_173 (get_defense_ranking): NBA best-defence ranking by season.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 425 OUT94

✓

SelectFindAthleteToolBFCL/multiple_172 (sports_db.find_athlete): athlete profile by name/sport.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 410 OUT343

✓

SelectFindCardToolBFCL/multiple_178 (find_card_in_deck): find a card by rank/suit in default deck.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 421 OUT66

✓

SelectGameWinnerToolBFCL/multiple_171 (game_result.get_winner): winner of a game between two teams on a date.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 440 OUT111

✓

SelectPokerFullHouseToolBFCL/multiple_179 (poker_probability.full_house): probability of a full house (default deck/hand).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 411 OUT140

✓

SelectSoccerPlayerStatsToolBFCL/multiple_170 (soccer_stat.get_player_stats): player stats by name/season.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 438 OUT345

✓

SelectSportsRankingToolBFCL/multiple_174 (sports_ranking): team ranking in a league.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 408 OUT200

×

SelectTeamScheduleToolBFCL/multiple_176 (sports_team.get_schedule): team schedule for N games in a league.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN20 356 OUT291

✓

SelectTopPlayerToolBFCL/multiple_175 (sports_ranking.get_top_player): top player by sport/gender.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 409 OUT92

Tool Calling - Tool Selection: Statistics, Finance & Health 6 passed

100.0%

IN56 430 OUT1 363

✓

SelectBmiMetersToolBFCL/multiple_117 (calculate_BMI): BMI in kg/m, not soccer / hotel_booking distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN9 371 OUT248

✓

SelectGcdToolBFCL/multiple_101 (math.gcd): greatest common divisor, not get_top_cases.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN9 348 OUT77

✓

SelectHotelPricingToolBFCL/multiple_94 (hotel_room_pricing.get): hotel room pricing, not car/flight pricing.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 383 OUT154

✓

SelectReturnOnEquityToolBFCL/multiple_128 (calculate_return_on_equity): ROE from income/equity/dividends, not library/lyrics/law distractors. dividends_paid is required 200000: prompt explicitly supplies "dividends paid of $200,000".1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 449 OUT292

✓

SelectSingleProbabilityToolBFCL/multiple_124 (probabilities.calculate_single): single-event probability, not math.power / fetch_DNA_sequence.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 433 OUT328

✓

SelectTwoSampleTtestToolBFCL/multiple_125 (run_two_sample_ttest): two-group t-test, not restaurant/personality distractors. equal_variance is required true: prompt says "assuming equal variance" (ground truth = [true, ""]).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 446 OUT264

Tool Calling - Tool Selection: Statistics, Finance & Services 8 passed

100.0%

IN91 297 OUT1 573

✓

SelectBmiMetricToolBFCL/multiple_116 (calculate_bmi): BMI in kg/cm with unit option, not celebrity_net_worth.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 399 OUT115

✓

SelectCalculateProbabilityToolBFCL/multiple_123 (calculate_probability): probability from favorable/total, not lawsuit/house-price distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 371 OUT193

✓

SelectCompoundInterestToolBFCL/multiple_129 (compound_interest): compound interest, not lawsuits_search.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 426 OUT210

✓

SelectFindRestaurantsToolBFCL/multiple_115 (find_restaurants): restaurant search with count + dietary, not instrument/stock/sports.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 404 OUT233

✓

SelectLinearRegressionToolBFCL/multiple_122 (run_linear_regression): regression with predictors, not recipe/itinerary/cooking distractors. standardize is required true: prompt says "Also apply standardization" (ground truth = [true]).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN11 431 OUT147

✓

SelectMoviesByActorToolBFCL/multiple_118 (imdb.find_movies_by_actor): movies by actor+year, not flight.book / lawsuits_search.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 393 OUT253

✓

SelectQuarterlyDividendToolBFCL/multiple_127 (finance.calculate_quarterly_dividend_per_share): dividend per share, not lyrics/paint/movie distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN11 425 OUT146

✓

SelectTTestToolBFCL/multiple_126 (t_test): two-dataset t-test, not event_finder / triangle-area distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 448 OUT276

Tool Calling - Tool Selection: Travel & Biology 7 passed1 failed

87.5%

IN104 343 OUT1 352

✓

SelectBookHotelToolBFCL/multiple_190 (book_hotel): book a room by hotel/location/room_type/start/nights.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN12 285 OUT137

×

SelectCellProteinsToolBFCL/multiple_196 (cellbio.get_proteins): proteins in a cell compartment.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN18 600 OUT461

✓

SelectDistanceDurationToolBFCL/multiple_193 (maps.get_distance_duration): distance/duration between two places.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 235 OUT142

✓

SelectForestGrowthToolBFCL/multiple_199 (forest_growth_forecast): forest growth forecast with human impact.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 238 OUT147

✓

SelectGenotypeFrequencyToolBFCL/multiple_198 (calculate_genotype_frequency): genotype frequency from allele frequency.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 246 OUT130

✓

SelectHeatCapacityToolBFCL/multiple_195 (calc_heat_capacity): heat capacity for a gas by temp/volume. Tightened per methodology: prompt explicitly states the gas is "air", so gas is required exactly "air" (ground truth also accepts omission, which equals the schema default).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 262 OUT96

✓

SelectMuseumHoursToolBFCL/multiple_194 (get_museum_hours): museum opening hours on a day.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 227 OUT131

✓

SelectMutationTypeToolBFCL/multiple_197 (mutation_type.find): gene mutation type by SNP id.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 250 OUT108

Tool Calling - Tool Selection: Triangle Properties 1 passed

100.0%

IN4 617 OUT277

✓

SelectTrianglePropertiesToolBFCL/multiple_0 (triangle_properties.get): retrieves triangle dimensions/properties from 3 sides.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 617 OUT277

Qwen 3.6 27B FP8 Instruct | Benchmark #45

Tool Calling Benchmark based on BFCL

Results by difficulty

Results by category