Ai4Users - Qwen 3.8 27B FP8 Think: medium

Benchmark items
Tool Calling - Irrelevance: Architecture & Artwork 9 passed	100.0%	IN31 133 OUT4 738
✓	NoToolForApartmentRentCostBFCL/irrelevance_158: apartment rental cost asked; only a new-construction cost estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 461 OUT386
✓	NoToolForBestJazzSingerBFCL/irrelevance_164: best Jazz singer asked; only a top-instrument-players search (singers are not instrument players) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 456 OUT474
✓	NoToolForCathedralCeilingDesignBFCL/irrelevance_157: how to design a cathedral ceiling asked; only a lookup of facts about an existing building is available (retrieves data, does not advise on design) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 457 OUT865
✓	NoToolForHolyBookDefinitionBFCL/irrelevance_144: definition of a holy book asked; only a search-within-a-named-holy-book tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 455 OUT352
✓	NoToolForLectureTeacherBFCL/irrelevance_167: teacher of upcoming lectures asked; only an upcoming-concert details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 458 OUT280
✓	NoToolForMostFamousSculptureBFCL/irrelevance_153: most famous sculpture asked; only a sculpting-tools lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 463 OUT523
✓	NoToolForPianoMaintenanceTipsBFCL/irrelevance_166: piano maintenance tips asked; only an instrument rental-prices lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 458 OUT573
✓	NoToolForScreamArtistBFCL/irrelevance_159: artist behind 'The Scream' asked. NOTE: the only tool (artwork_search) requires a museum_location that the user did not provide, so it cannot be correctly called; BFCL labels this irrelevant. Kept faithfully with ground truth = no tool call (flagged as borderline).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 462 OUT397
✓	NoToolForSculptureRockTypeBFCL/irrelevance_152: which rock suits a garden sculpture asked; only a sculpture 3D-model creator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 463 OUT888
Tool Calling - Irrelevance: Buildings, Sculpture & Music 7 passed1 skipped	100.0%	IN22 989 OUT3 136
–	NoToolForCelloInstrumentTypeBFCL/irrelevance_165: instrument classification of a cello asked. NOTE: BFCL labels this irrelevant, but the only tool (get_instrument_info with detail=type) arguably answers it; flagged for review as a borderline BFCL item. Kept faithfully with ground truth = no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_instrument_info' with detail='type' answers 'what type of instrument is a cello' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForDeclarationSignersBFCL/irrelevance_137: who signed the declaration asked; only a historical-event DATE lookup is available (returns a date, not the signatories) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 282 OUT519
✓	NoToolForEiffelTowerHeightBFCL/irrelevance_156: Eiffel Tower height asked; only an architecture-plan generator (creates a new design, does not retrieve facts) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 286 OUT307
✓	NoToolForFamousDancersBFCL/irrelevance_162: famous dancers asked; only a museum-visual-artists lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 288 OUT514
✓	NoToolForReformationInitiatorBFCL/irrelevance_145: who initiated the Reformation asked; only a religious-event YEAR lookup is available (returns a year, not a person) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 281 OUT469
✓	NoToolForScientistDefinitionBFCL/irrelevance_143: definition of "scientist" asked; only a named-historical-figure info lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 278 OUT406
✓	NoToolForSellingGuitarBFCL/irrelevance_163: how to sell a guitar asked; only an instrument-tuning tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 283 OUT498
✓	NoToolForSimilarBlueColorBFCL/irrelevance_148: a color similar to blue asked; only a complementary-color tool (opposite, not similar) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 291 OUT423
Tool Calling - Irrelevance: Cell Biology & Genetics 8 passed2 skipped	100.0%	IN30 315 OUT5 490
✓	NoToolForClottingBloodCellsBFCL/irrelevance_32: blood cell type responsible for clotting asked; only a nucleus-count-based cell-type classifier is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 790 OUT607
✓	NoToolForCommonAncestorBFCL/irrelevance_41: common ancestor of two animals asked (phylogenetics knowledge); only a fibonacci-sequence calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 789 OUT667
✓	NoToolForEvolutionaryAlgorithmInGameAIBFCL/irrelevance_43: how to apply an Evolutionary Algorithm in game AI asked (advice); only a creature-evolution simulator is available (wrong context).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 790 OUT1 173
✓	NoToolForGeneVariantFrequencyBFCL/irrelevance_36: gene-variant frequency in a population asked; only a dominant-trait-from-alleles tool is available (the true-match variant-frequency tool is deliberately isolated in A4).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 795 OUT666
–	NoToolForIdentifyingGeneticSequenceBFCL/irrelevance_33: interpret a genetic code sequence asked; only a species-identification-from-sequence tool is available (returns a species, not what the sequence means).0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'identify_species' (identifies a species from a genetic sequence) plausibly answers the query about a DNA sequence, so a competent model may call it - BFCL-mislabeled/borderline-relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForIncreasingDeerPopulationBFCL/irrelevance_39: how-to advice on growing an animal population asked; only a population-growth CALCULATOR (needs numeric rates) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 789 OUT718
✓	NoToolForLionMatingProcessBFCL/irrelevance_35: mating process of a species asked; only a dominant/recessive genetic-traits lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 786 OUT529
✓	NoToolForMitochondriaCountBFCL/irrelevance_29: number/count of mitochondria asked; only a cell-part FUNCTION lookup is available (wrong query direction).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 790 OUT526
✓	NoToolForPandaEvolutionaryHistoryBFCL/irrelevance_42: evolutionary history asked (a narrative fact); only a biodiversity-index calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 786 OUT604
–	NoToolForWhaleGeneSequenceBFCL/irrelevance_44: the actual gene sequence of a species asked; only a gene-sequence GENERATOR (needs numeric mutation rate + duration) is available.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'gene_sequencer' (generate gene sequences for evolutionary changes) directly answers the query - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
Tool Calling - Irrelevance: Color, Geometry & Population 11 passed	100.0%	IN40 836 OUT4 001
✓	NoToolForColorInComputerVisionBFCL/irrelevance_131: color meaning in computer vision asked; only a color-in-personality-psychology analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 712 OUT616
✓	NoToolForDeclarationSigningDateBFCL/irrelevance_135: signing date asked; only date-arithmetic (add days) and age tools are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 710 OUT276
✓	NoToolForElectricityDiscovererBFCL/irrelevance_140: who discovered electricity asked; only an electric-current calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 706 OUT310
✓	NoToolForHexColorMixBFCL/irrelevance_146: mixing two hex colors asked; only a prophet-details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 722 OUT561
✓	NoToolForIndependenceSigningDateBFCL/irrelevance_138: signing date asked; only an age calculator (needs birthdate + current date) and a date-adder are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 710 OUT300
✓	NoToolForMostVisitedMarketBFCL/irrelevance_161: most visited market asked; only a museum visitation-statistics lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 712 OUT334
✓	NoToolForNbaFinalWinnerBFCL/irrelevance_133: NBA final winner asked; only a historical-battle details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 713 OUT250
✓	NoToolForPicassoPaintingYearBFCL/irrelevance_151: year of a Picasso painting asked; only a sculpture-dimensions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 711 OUT316
✓	NoToolForRelativityAuthorBFCL/irrelevance_142: who proposed special relativity asked; only a chemical-element properties lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 715 OUT253
✓	NoToolForSculptureCreatorBFCL/irrelevance_155: sculpture creator asked; only a max-seating-capacity calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 711 OUT241
✓	NoToolForWhichColorsToMixBFCL/irrelevance_150: which colors to mix for a shade asked; only an RGB-to-Pantone converter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 714 OUT544
Tool Calling - Irrelevance: Cooking, Currency & Markets 8 passed	100.0%	IN25 943 OUT4 371
✓	NoToolForBakerySourdoughBFCL/irrelevance_223: a BAKERY selling sourdough asked; only a grocery-shop product finder available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 245 OUT805
✓	NoToolForChickenRecipeBFCL/irrelevance_211: the RECIPE to cook chicken asked; only a cook-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 241 OUT640
✓	NoToolForChocolateCakeIngredientsBFCL/irrelevance_220: cake INGREDIENTS asked; only a cooking-TEMPERATURE tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 244 OUT475
✓	NoToolForCurrencyToGbpBFCL/irrelevance_233: CURRENCY conversion asked; only a WEATHER tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 246 OUT253
✓	NoToolForHowToBoilAnEggBFCL/irrelevance_212: best WAY to boil an egg asked; only a boiling-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 243 OUT697
✓	NoToolForLasagnaRecipeBFCL/irrelevance_219: the RECIPE for lasagna asked; only a recipe-CALORIE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 241 OUT604
✓	NoToolForLegExercisesBFCL/irrelevance_221: leg EXERCISES asked; only a grocery-list generator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 241 OUT515
✓	NoToolForPetStoreBFCL/irrelevance_224: a PET store asked; only a GROCERY-store locator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 242 OUT382
Tool Calling - Irrelevance: Dates & Elements 10 passed	100.0%	IN36 270 OUT4 213
✓	NoToolForHydrogenPropertiesBFCL/irrelevance_141: chemical properties of Hydrogen asked; only a scientist-contributions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 626 OUT748
✓	NoToolForLargestPlanetBFCL/irrelevance_139: largest planet asked; only a star-info lookup (needs a star name) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 626 OUT364
✓	NoToolForMostImportantProphetBFCL/irrelevance_147: most important prophet asked; only a two-color mixer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 626 OUT296
✓	NoToolForMuseumVisitFrequencyBFCL/irrelevance_160: museum member visit frequency asked; only a single most-frequent-visitor lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 631 OUT314
✓	NoToolForPantoneOfSkyBlueBFCL/irrelevance_149: Pantone code of sky blue asked; only a paint-proportions calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 628 OUT449
✓	NoToolForPersonalityTypeBFCL/irrelevance_130: user asks what personality type they are; only a big-five calculator (which needs the user's rated answers) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 624 OUT371
✓	NoToolForStadiumSeatingCapacityBFCL/irrelevance_154: stadium seating capacity asked; only a sculpture-creator lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 627 OUT328
✓	NoToolForVicePresidentBFCL/irrelevance_136: Vice President asked; only a US-president-by-year lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 626 OUT375
✓	NoToolForWaterlooCasualtiesBFCL/irrelevance_132: casualty count of a battle asked; only a historical-event date lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 628 OUT650
✓	NoToolForWorldCupWinnerBFCL/irrelevance_134: sports World Cup winner asked; only a historical-battle outcome predictor is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 628 OUT318
Tool Calling - Irrelevance: Electromagnetism, Thermo & Data 9 passed1 failed	90.0%	IN37 554 OUT4 775
✓	NoToolForElectronRestMassBFCL/irrelevance_23: mass of an electron asked (a knowledge fact); only magnetic-field / current tools are available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 397 OUT288
✓	NoToolForFlightDurationBFCL/irrelevance_20: flight duration asked; only a currency converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 400 OUT313
✓	NoToolForFreezingPointBFCL/irrelevance_25: freezing point asked; only boiling-point calculators are available (wrong phase transition).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 407 OUT672
×	NoToolForGasAmountGeneratedBFCL/irrelevance_26: amount of gas generated asked; only a gas-PRESSURE calculator is available (wrong quantity).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 929 OUT1 231
✓	NoToolForGiniCoefficientBFCL/irrelevance_48: Gini coefficient asked; only a GDP-fetch tool is available (different economic indicator).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 400 OUT404
✓	NoToolForHeatingWaterBFCL/irrelevance_27: heat energy to warm water asked. NOTE: the original tool (calculate_heat) is a fair match for this query (only the well-known specific heat is missing), which would make "no call" ambiguous; per human-baseline fairness it is deliberately NOT shown here (kept as a distractor in A3), so no visible tool fits and "no call" is unambiguously correct.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 411 OUT512
✓	NoToolForHexagonSidesBFCL/irrelevance_28: number of sides of a hexagon asked (basic geometry knowledge); only a boiling-point tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 398 OUT45
✓	NoToolForSymbolicMagneticFieldBFCL/irrelevance_22: magnetic field asked with SYMBOLIC values (distance 'r', current 'I'); the two magnetic-field tools require concrete numeric distance + current, so there is nothing to pass -> no fit.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 411 OUT589
✓	NoToolForTimeDifferenceBFCL/irrelevance_49: time-zone difference asked; only a geo-coordinate lookup tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 400 OUT497
✓	NoToolForWorldSeriesWinnerBFCL/irrelevance_53: sports trivia asked; only a SQL database-query runner is available (no database/query provided).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 401 OUT224
Tool Calling - Irrelevance: Entertainment, Sports & Knowledge 9 passed	100.0%	IN31 410 OUT2 781
✓	NoToolForBasketballResultBFCL/irrelevance_84: basketball game result asked; only a stock-price retrieval tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 490 OUT220
✓	NoToolForBestSoccerPlayerBFCL/irrelevance_79: best current soccer player asked; only a stock-market prediction tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 488 OUT226
✓	NoToolForCaliforniaGovernorBFCL/irrelevance_87: current Governor of California asked; only a court-case filing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 486 OUT282
✓	NoToolForCrimeThrillerMovieListBFCL/irrelevance_88: best crime-thriller movies of 2020 asked; only a document forgery detector is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 495 OUT520
✓	NoToolForFifaWorldCupWinnerBFCL/irrelevance_80: FIFA World Cup 2010 winner asked; only a stock-ticker lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 491 OUT142
✓	NoToolForFrenchTranslationBFCL/irrelevance_61: French translation of a word asked; only an SVM-model tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 490 OUT140
✓	NoToolForHorrorMovieListBFCL/irrelevance_77: horror movie recommendations asked; only an investment future-value tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 489 OUT554
✓	NoToolForPresidentialElectionWinnerBFCL/irrelevance_85: 2020 presidential election winner asked; only a criminal-case detail lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 491 OUT237
✓	NoToolForSportsNewsBFCL/irrelevance_76: latest sports news asked; only a portfolio investment-trend analyzer is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 490 OUT460
Tool Calling - Irrelevance: Finance & Economics 8 passed	100.0%	IN26 826 OUT3 526
✓	NoToolForAverageHousePriceBFCL/irrelevance_69: market-wide average house price asked; only a regression predictor (needs a trained model + feature vector, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 346 OUT280
✓	NoToolForCompoundInterestBFCL/irrelevance_70: compound-interest computation asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 372 OUT834
✓	NoToolForGoldPriceBFCL/irrelevance_78: today's gold spot price asked; only a bond-price calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 348 OUT420
✓	NoToolForHorrorMovieCatalogBFCL/irrelevance_81: horror movie recommendations asked; only a historical stock-price fetcher is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 349 OUT585
✓	NoToolForLossProjectionBFCL/irrelevance_73: a loss projection asked; only a revenue-prediction tool is available (revenue != loss).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 351 OUT331
✓	NoToolForPoundToKilogramBFCL/irrelevance_75: a mass unit conversion asked; only a portfolio-returns calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 347 OUT126
✓	NoToolForProfitMarginBFCL/irrelevance_71: profit margin asked; only an ROI calculator is available (ROI != margin) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 369 OUT523
✓	NoToolForStockNewsBFCL/irrelevance_82: stock news asked; only a capital-gains calculator is available (gains != news) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 344 OUT427
Tool Calling - Irrelevance: Finance Near-Matches 9 passed	100.0%	IN30 921 OUT4 025
✓	NoToolForBitcoinPriceBFCL/irrelevance_56: current bitcoin price asked; only a generic database record-select tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 426 OUT302
✓	NoToolForBookstoreLookupBFCL/irrelevance_55: which online bookstore sells a title asked; only an add-to-cart tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 431 OUT302
✓	NoToolForBusinessRateOfReturnBFCL/irrelevance_74: rate of return asked; only a net-profit calculator is available (profit != rate of return).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 447 OUT666
✓	NoToolForCoinFlipProbabilityBFCL/irrelevance_66: probability of a coin landing heads asked; only a standard-deviation tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 431 OUT221
✓	NoToolForCurrentInterestRateBFCL/irrelevance_83: the current market interest rate asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 425 OUT306
✓	NoToolForExternalRateOfReturnBFCL/irrelevance_72: EXTERNAL rate of return asked; only an internal-rate-of-return (IRR) tool is available (ERR != IRR, even though cash flows are given) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 455 OUT518
✓	NoToolForFinalPriceWithoutBasePriceBFCL/irrelevance_60: final price after discount+tax asked, but the original price is never given, so the (domain-matching) calculateFinalPrice cannot be correctly called -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 445 OUT264
✓	NoToolForFrontendDevelopmentBFCL/irrelevance_62: how to build an e-commerce frontend asked; only a recommender-model builder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 431 OUT1 078
✓	NoToolForHighestGrossingMovieBFCL/irrelevance_54: highest-grossing movie of all time asked; only a movie search (requires a specific title + year, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 430 OUT368
Tool Calling - Irrelevance: Food, Time & Distance 4 passed1 skipped	100.0%	IN10 832 OUT1 175
✓	NoToolForCakeBakingMachineBFCL/irrelevance_218: a baking MACHINE/appliance asked; only a recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 706 OUT401
✓	NoToolForDrivingTravelTimeBFCL/irrelevance_215: driving TRAVEL TIME asked; only a restaurant-bill tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 710 OUT261
✓	NoToolForLondonTimezoneNameBFCL/irrelevance_226: the NAME of London's timezone asked; only a local-TIME-by-timezone tool available (it needs the timezone as input and returns a time, not the timezone name) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 702 OUT349
–	NoToolForPizzaInBostonBFCL/irrelevance_213: a place for pizza asked; a cuisine+location restaurant_finder is available. REVIEW FLAG (fairness): restaurant_finder(cuisine=pizza/Italian, location=Boston) plausibly answers this, so a competent human might legitimately call it. BFCL labels it irrelevant; kept as such and marked Hard because the intended "no call" is debatable. No OTHER query here is a restaurant search, so the tool cannot wrongly satisfy the file's other cases.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'restaurant_finder' (cuisine+location) answers 'good place for pizza in Boston' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForTriangleHypotenuseBFCL/irrelevance_238: the HYPOTENUSE of a triangle asked; only a geo lat/long distance calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 714 OUT164
Tool Calling - Irrelevance: Geo, Travel & Sentiment 7 passed1 failed1 skipped	87.5%	IN30 622 OUT3 352
✓	NoToolForAverageTemperatureBmiBFCL/irrelevance_110: an average December temperature asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 396 OUT296
✓	NoToolForCaseLawyerBFCL/irrelevance_95: the lawyer of a case asked; only a case-outcome retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 399 OUT428
✓	NoToolForCurrentTimeSunriseBFCL/irrelevance_102: the current time asked; only a sunrise/sunset time calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 395 OUT274
✓	NoToolForFamousQuoteBFCL/irrelevance_119: a famous quote asked; only a sentiment-analysis tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 393 OUT93
✓	NoToolForFastestRouteBFCL/irrelevance_114: the fastest route asked; only a closest-airport finder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 397 OUT697
–	NoToolForRestaurantsInLondonBFCL/irrelevance_113: restaurants in London asked; only a generic points-of-interest locator is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'find_pois' (locate points of interest by category) answers 'restaurants in London' (restaurants are POIs) - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForSportsBrainImpactBFCL/irrelevance_120: the neurological impact of sports asked; only a caffeine neurological-impact tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 396 OUT455
✓	NoToolForTrafficConditionBFCL/irrelevance_112: the current traffic condition asked; only an address geocoder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 395 OUT371
×	NoToolForTravelTimeByCarBFCL/irrelevance_115: travel time by car asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 851 OUT738
Tool Calling - Irrelevance: Legal & Case Lookup 10 passed	100.0%	IN35 507 OUT3 594
✓	NoToolForAppleCaloriesBFCL/irrelevance_100: calories in an apple asked; only a litigation-cost calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 548 OUT261
✓	NoToolForBrainWeightBFCL/irrelevance_122: average human brain weight asked; only a per-patient neural-activity tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 550 OUT236
✓	NoToolForCaseJudgementBFCL/irrelevance_93: a case judgement asked; only a law-firm impactful-cases lister is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 548 OUT328
✓	NoToolForChineseRestaurantBFCL/irrelevance_97: a Chinese-restaurant recommendation asked; only a lawsuit-filing tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 551 OUT396
✓	NoToolForMovieRecommendationBFCL/irrelevance_116: a movie recommendation asked; only a word-count tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 549 OUT358
✓	NoToolForNbaChampionBFCL/irrelevance_118: the 1996 NBA champion asked; only an NBA playoff-brackets display tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 551 OUT506
✓	NoToolForPaintingEiffelSettlementBFCL/irrelevance_98: how long to paint the Eiffel Tower asked; only a lawsuit-settlement estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 552 OUT244
✓	NoToolForSecurityGuardSearchBFCL/irrelevance_92: finding a security guard asked; only a lawyer-search tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 558 OUT456
✓	NoToolForSphereVolumeBFCL/irrelevance_104: sphere volume asked; only a circular-park area tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 552 OUT378
✓	NoToolForTrafficLawsBFCL/irrelevance_99: traffic laws asked; only a lawsuit-search tool is available (lawsuits, not laws) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 548 OUT431
Tool Calling - Irrelevance: Math & Science Distractors 8 passed	100.0%	IN25 236 OUT2 116
✓	NoToolForDefiniteIntegralBFCL/irrelevance_9: definite integral asked; only an income-tax calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 157 OUT324
✓	NoToolForIntegralOfCubeBFCL/irrelevance_8: definite integral asked; only a string-to-int converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 150 OUT319
✓	NoToolForLatestMoviesBFCL/irrelevance_15: latest movie releases asked; only a physics final-velocity tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 144 OUT311
✓	NoToolForPerpendicularSlopeBFCL/irrelevance_3: slope of a perpendicular line asked; only a critical-points finder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 162 OUT217
✓	NoToolForPrimeFactorsBFCL/irrelevance_13: prime factorization asked; only a compound-interest calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 147 OUT255
✓	NoToolForRectangleAreaBFCL/irrelevance_6: rectangle area asked; only a circle-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 158 OUT106
✓	NoToolForTriangleAreaBFCL/irrelevance_0: triangle area asked; only a BMI tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 158 OUT247
✓	NoToolForUpwardAccelerationBFCL/irrelevance_14: acceleration asked (it is just gravity); the tempting max-height and final-velocity tools compute neither the acceleration nor what is asked -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 160 OUT337
Tool Calling - Irrelevance: Math & Statistics 5 passed1 failed	83.3%	IN21 146 OUT3 224
✓	NoToolForBinomialMeanBFCL/irrelevance_67: the MEAN of a binomial experiment asked; only p-value tools are available (they return a p-value, not the mean) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 026 OUT783
✓	NoToolForCivilLawComponentsBFCL/irrelevance_90: the components/categories of Civil Law asked; only a noise-complaint filing tool is available (filing a complaint does not explain law) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN2 998 OUT575
✓	NoToolForCoinTossMaxHeadsBFCL/irrelevance_63: how many heads CAN be obtained from tossing 3 coins (a count/maximum, not a probability) asked; only a probability-of-an-event calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 002 OUT176
×	NoToolForNullHypothesisStdDevBFCL/irrelevance_68: the STANDARD DEVIATION of a null-hypothesis test asked; only p-value tools are available (they return a p-value, not a standard deviation) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 112 OUT892
✓	NoToolForPowerOfTwoBFCL/irrelevance_59: raising a number to a power asked; only a linear-equation solver is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 001 OUT155
✓	NoToolForQuadraticEquationBFCL/irrelevance_57: solving a quadratic equation asked; only a genetic-algorithm optimizer and a LINEAR equation solver are available (a quadratic is not linear) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 007 OUT643
Tool Calling - Irrelevance: Medical, Social & Personality 6 passed3 skipped	100.0%	IN20 452 OUT2 663
✓	NoToolForBigMacCaloriesBFCL/irrelevance_123: calories of a Big Mac asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 407 OUT275
✓	NoToolForEconomicHappinessEffectBFCL/irrelevance_126: the effect of economic status on happiness asked; only a happiness-index lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 409 OUT569
✓	NoToolForFriendMbtiBFCL/irrelevance_129: a friend's MBTI asked; only a personality-type evaluator (needs trait percentages) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 407 OUT369
–	NoToolForLatestTechTrendBFCL/irrelevance_124: the latest technology trend asked; only a trending-topics-by-category tool is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_social_trends' (trending topics by category) answers 'latest trend in technology' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForMajorAirportsBFCL/irrelevance_117: major US airports asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 410 OUT490
–	NoToolForMotorNeuronDiseaseBFCL/irrelevance_121: motor neuron disease info asked; only a generic disease-info retriever is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'medical_records_get_disease_info' directly answers 'information on motor neuron diseases' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForPopularBooksBFCL/irrelevance_125: popular books by an author asked; only a recent-tweets retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 409 OUT574
–	NoToolForTwitterMoodBFCL/irrelevance_127: overall twitter mood asked; only a twitter sentiment-analysis tool is available -> BFCL ground truth is no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'sentiment_analysis_twitter' directly answers 'general mood of twitter regarding the new iPhone' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForVegetableServingsBFCL/irrelevance_128: daily vegetable servings asked; only a personality-test scorer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 410 OUT386
Tool Calling - Irrelevance: Mixed Distractors 8 passed	100.0%	IN25 992 OUT2 905
✓	NoToolForAirQualityBFCL/irrelevance_40: current air quality asked; only a plant-biomass calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 250 OUT302
✓	NoToolForFavoriteRestaurantBFCL/irrelevance_31: a favorite restaurant recommendation asked; only a cell-division simulator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 246 OUT264
✓	NoToolForGlobalWarmingDocumentaryBFCL/irrelevance_38: a documentary (video) asked; only a scientific-research-paper retriever is available (wrong medium).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 247 OUT411
✓	NoToolForLasagnaIngredientsBFCL/irrelevance_47: recipe ingredients asked; only a flight-timetable tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 247 OUT508
✓	NoToolForLionDominantTraitBFCL/irrelevance_34: dominant genetic trait of a species asked; only a gene-variant-FREQUENCY lookup is available (different genetic output).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 249 OUT579
✓	NoToolForMultinucleatedCellNameBFCL/irrelevance_30: name of a cell type with multiple nuclei asked; only a blood-cell classifier (by shape/size) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 253 OUT362
✓	NoToolForPicnicSpotBFCL/irrelevance_37: a picnic spot asked; only a local-fauna information tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 246 OUT378
✓	NoToolForTranslationBFCL/irrelevance_52: natural-language translation asked; only a file-compression tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 254 OUT101
Tool Calling - Irrelevance: Motion, Electricity & Misc 10 passed	100.0%	IN35 408 OUT5 124
✓	NoToolForCarProjectileRangeBFCL/irrelevance_16: projectile-style "how far will a car travel" asked with symbolic v/t/theta; the range tool needs concrete numeric velocity + angle -> nothing to pass -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 551 OUT693
✓	NoToolForConeVolumeBFCL/irrelevance_19: cone volume asked; only an investment-return calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 545 OUT412
✓	NoToolForCurrentClockTimeBFCL/irrelevance_17: current clock time asked; only a travel-time (distance/speed) calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 534 OUT241
✓	NoToolForElectronMassBFCL/irrelevance_24: mass of an electron asked (a knowledge fact); only an electric-current tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 535 OUT290
✓	NoToolForFibonacciOf20BFCL/irrelevance_51: fibonacci number asked; only a cryptocurrency-price tool is available (the fibonacci calculator lives in a different file so it cannot satisfy this query).1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 536 OUT656
✓	NoToolForForceAngleFromMomentumBFCL/irrelevance_18: angle of a force from momentum asked; only a vector-angle-from-XY-components tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 541 OUT1 120
✓	NoToolForHorrorMovieListBFCL/irrelevance_50: movie recommendations asked; only a Celsius-to-Fahrenheit converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 541 OUT563
✓	NoToolForMagneticFieldNearWireBFCL/irrelevance_21: magnetic field near a current-carrying wire asked; only an EM-wave-amplitude tool is available (magnetic-field calculators are deliberately kept in a different file).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 550 OUT541
✓	NoToolForSineOf45DegreesBFCL/irrelevance_45: sine of an angle asked; only a polygon-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 536 OUT286
✓	NoToolForTeslaPriceBFCL/irrelevance_46: product price of a car asked; only a currency-exchange-rate tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 539 OUT322
Tool Calling - Irrelevance: Music Theory & Lookup 8 passed	100.0%	IN25 572 OUT3 889
✓	NoToolForAncientGreekLawmakersBFCL/irrelevance_171: who established laws in Ancient Greece asked (history); only a musical-piece composer search is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 194 OUT721
✓	NoToolForBasketballGameAtVenueBFCL/irrelevance_169: who is playing a basketball game at a venue asked; only concert-locating tools exist.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 197 OUT321
✓	NoToolForBluesScalePatternBFCL/irrelevance_176: the interval pattern of a blues scale asked; only composer lookups are available (get_blues_scale lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 199 OUT715
✓	NoToolForDonQuixoteAuthorBFCL/irrelevance_172: author of the novel Don Quixote asked; only music-composition lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 192 OUT349
✓	NoToolForFamousUsComposersBFCL/irrelevance_170: most famous US composers asked (general knowledge, no specific piece); only a chord-progression creator + piece-specific composer lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 195 OUT502
✓	NoToolForGrammyBestAlbumBFCL/irrelevance_177: Grammy Award winner asked; only a song chord-progression lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 200 OUT528
✓	NoToolForPrimaryTriadsOfKeyBFCL/irrelevance_173: primary triads of a key asked; only a genre-common-chords finder is available (primary_triads lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 198 OUT441
✓	NoToolForUniversityClassAvailabilityBFCL/irrelevance_168: availability of a university class asked; only concert-availability / concert-search tools are available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 197 OUT312
Tool Calling - Irrelevance: Probability & Legal 5 passed	100.0%	IN13 997 OUT3 025
✓	NoToolForFaceCardProbabilityBFCL/irrelevance_64: probability of drawing a face card from a deck asked; only a COIN-toss heads probability tool is available (coins, not cards) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 796 OUT351
✓	NoToolForHomeElectricityNeedBFCL/irrelevance_58: home electricity demand (kWh) asked; only a solar-panel COUNT calculator is available (it sizes panels, not electricity need, and needs sunlight hours not given) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN2 799 OUT719
✓	NoToolForNoiseComplaintProcessBFCL/irrelevance_91: how/whether to report a noise complaint asked; only a law-category listing tool is available (it lists legal categories, it does not file or advise on complaints) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 796 OUT481
✓	NoToolForRedMarbleCountBFCL/irrelevance_65: number of red marbles asked; only a determine-population tool is available (it returns a total population from probability + sample size, not a sub-category count) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN2 811 OUT530
✓	NoToolForTenantRightsBFCL/irrelevance_89: explaining tenant rights asked; only a contract-generation tool is available (generating a contract does not explain rights) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN2 795 OUT944
Tool Calling - Irrelevance: Recipes & Cooking 8 passed1 skipped	100.0%	IN27 642 OUT2 870
✓	NoToolForBestGameCharacterBFCL/irrelevance_206: best game character vs a dragon asked; only a best-WEAPON finder available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 459 OUT445
✓	NoToolForBestMovie2020BFCL/irrelevance_235: best MOVIE of 2020 asked; only a currency-exchange calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 456 OUT299
✓	NoToolForBuildingAPcBFCL/irrelevance_209: how to build a PC asked; only a cooking-recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 451 OUT631
✓	NoToolForFamousPlaceInParisBFCL/irrelevance_210: most famous PLACE in Paris asked; only a restaurant-by-dish search available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 453 OUT400
✓	NoToolForGameShortestPathBFCL/irrelevance_208: shortest in-game path asked; only a taxi-fare calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 461 OUT389
✓	NoToolForInchToMeterBFCL/irrelevance_234: LENGTH conversion (inches to metres) asked; only a CURRENCY converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 452 OUT232
✓	NoToolForLowestGameScoreBFCL/irrelevance_207: LOWEST game score asked; only a HIGHEST-score retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 456 OUT211
✓	NoToolForNewYorkPopulationBFCL/irrelevance_225: current POPULATION of New York asked; only a time-zone converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 454 OUT263
–	NoToolForQuickestFlightLondonTokyoBFCL/irrelevance_236: quickest way to fly London->Tokyo asked; a flight-duration tool is available. REVIEW FLAG (fairness): get_flight_duration plausibly answers this ("quickest ... by plane" ~= quickest flight duration), so a competent human might legitimately call it. Kept as BFCL labels it (irrelevant), but this item is borderline. Marked Hard because the intended "no call" is debatable.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_flight_duration' (with flight_type non-stop) plausibly answers 'quickest way ... by plane' - BFCL-mislabeled/borderline-relevant. Kept for traceability.	Skipped	IN0 OUT0
Tool Calling - Irrelevance: Restaurants & Travel 8 passed	100.0%	IN26 515 OUT2 531
✓	NoToolForBeijingSunsetBFCL/irrelevance_227: SUNSET time asked; only a SUNRISE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 313 OUT139
✓	NoToolForClosestItalianRestaurantBFCL/irrelevance_216: FIND the closest Italian restaurant asked; only a currency converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 312 OUT175
✓	NoToolForFindSushiRestaurantBFCL/irrelevance_214: FIND a sushi restaurant asked; only a tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 314 OUT242
✓	NoToolForNearestPharmacyBFCL/irrelevance_237: FIND the nearest pharmacy asked; only a route-to-a-known-location planner available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 314 OUT187
✓	NoToolForPetFriendlyTrainStationBFCL/irrelevance_230: a pet-friendly TRAIN STATION asked; only hotel search tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 313 OUT297
✓	NoToolForSushiRestaurantsTokyoBFCL/irrelevance_229: FIND sushi RESTAURANTS asked; only hotel-booking tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 314 OUT532
✓	NoToolForThaiRestaurantChicagoBFCL/irrelevance_231: FIND a Thai RESTAURANT asked; only a hotel-availability checker available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 315 OUT357
✓	NoToolForYogaRetreatBFCL/irrelevance_232: a yoga RETREAT asked; only a HOTEL search tool available. REVIEW FLAG (fairness): a retreat is not a hotel, but hotel_search_find_hotels accepts arbitrary amenities (spa, vegetarian) and a location, so it is somewhat tempting. BFCL labels this irrelevant (retreat != hotel); kept as such and marked Hard because the boundary is borderline.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 320 OUT602
Tool Calling - Irrelevance: Sports & Games (set 1) 10 passed	100.0%	IN35 688 OUT4 455
✓	NoToolForAssistAverageBFCL/irrelevance_185: assist average asked; only an average-SCORING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 570 OUT221
✓	NoToolForBestChessMoveBFCL/irrelevance_194: best next move asked; only a board-game-STATISTICS tool is available (the move-suggesting board_analyser lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 572 OUT451
✓	NoToolForChessArrangementCountBFCL/irrelevance_196: total theoretical arrangements asked; only a final-score calculator is available (possible_moves lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 572 OUT636
✓	NoToolForFootballTeamRankingBFCL/irrelevance_186: ranking of a team asked; only a season-MVP tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 568 OUT451
✓	NoToolForIceCreamStoreBFCL/irrelevance_201: find a physical store asked; only a video-game guide is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 565 OUT374
✓	NoToolForMostHeartsInPokerBFCL/irrelevance_199: who holds most of a suit asked; only a poker-dealing tool is available (get_highest_card_holder lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 572 OUT363
✓	NoToolForPlayerBiographyBFCL/irrelevance_189: a biographical "who is X" asked; only a top-ranked-athlete lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 565 OUT480
✓	NoToolForTeamRosterBFCL/irrelevance_192: full roster asked; only a top-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 567 OUT352
✓	NoToolForUnoRulesBFCL/irrelevance_198: the rules of a game asked; only a deck-shuffling tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 569 OUT665
✓	NoToolForWorldCupWinnerBFCL/irrelevance_191: who won a tournament asked (no team named); only a per-team match-stats tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 568 OUT462
Tool Calling - Irrelevance: Sports & Games (set 2) 8 passed1 failed	88.9%	IN34 674 OUT3 795
✓	NoToolForBlackjackAceRuleBFCL/irrelevance_200: a rules question asked; only a poker highest-card-holder tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 467 OUT649
✓	NoToolForBookAuthorBFCL/irrelevance_205: author of a book asked; only a video-game score calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 471 OUT67
✓	NoToolForChessTournamentWinnerBFCL/irrelevance_195: who won a tournament asked; only a board-position move-suggester is available (the game-history fetch_game_stats lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 468 OUT389
✓	NoToolForCreateJigsawBFCL/irrelevance_204: CREATE a puzzle asked; only a puzzle-SOLVING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 461 OUT377
×	NoToolForHaloPlayerRankBFCL/irrelevance_203: a player's RANK asked; only a player-SCORE lookup is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 947 OUT398
✓	NoToolForLakersTopScorerBFCL/irrelevance_193: top scorer of a team asked; only a team-details tool (roster/results/upcoming, not scorers) is available (sports_team.get_top_scorer lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 466 OUT634
✓	NoToolForMonopolyWinnerBFCL/irrelevance_197: who won a casual game asked; only a possible-moves calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 466 OUT386
✓	NoToolForTopTennisPlayerBFCL/irrelevance_190: top ranked PLAYER asked; only a TEAM standing/ranking tool is available (the athlete-ranking get_top_ranked lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 465 OUT305
✓	NoToolForWorldSeriesWinnerBFCL/irrelevance_202: who won a real game asked; only a score calculator (from raw points) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 463 OUT590
Tool Calling - Irrelevance: Sports Statistics 9 passed	100.0%	IN31 560 OUT4 351
✓	NoToolForBasketballSeasonMvpBFCL/irrelevance_187: who won the MVP asked; only a team-ranking tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 509 OUT273
✓	NoToolForClippersRosterBFCL/irrelevance_179: roster of a team asked; only a two-team game-results tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 502 OUT821
✓	NoToolForCommonChordsInPopBFCL/irrelevance_174: most common chords in a pop song asked; only a key-signature primary-triads tool is available (find_common_chords lives in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 507 OUT823
✓	NoToolForCricketMatchParticipantsBFCL/irrelevance_180: who is playing in cricket matches today asked; only a cricket-schedule tool (dates/matches, not participants) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 506 OUT452
✓	NoToolForLaLigaPlayersBFCL/irrelevance_181: who played in a whole league asked; only a two-named-team last-match-result tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 502 OUT418
✓	NoToolForMoonlightSonataComposerBFCL/irrelevance_175: composer of Moonlight Sonata asked; only a blues-scale generator is available here (all composer-lookup tools are isolated in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 505 OUT327
✓	NoToolForNbaMvpWinnerBFCL/irrelevance_184: who won an MVP award asked (no player named); only a per-player stats tool that requires a player name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 511 OUT614
✓	NoToolForTopAssistPlayerBFCL/irrelevance_178: top ASSIST player asked; only a top-GOAL-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 506 OUT231
✓	NoToolForWimbledonChampionBFCL/irrelevance_183: winner of a tournament asked (no person named); only a celebrity-info tool that requires a name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 512 OUT392
Tool Calling - Irrelevance: Time, Weather & Misc 2 passed1 failed1 skipped	66.7%	IN10 342 OUT1 745
×	NoToolForCityDistanceKmBFCL/irrelevance_239: DISTANCE (km) between two cities asked; only a get_date tool that returns the TIME difference between two locations is available (its 'unit' arg mentions distance, but it computes time) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN5 188 OUT900
–	NoToolForCurrentTimeSydneyBFCL/irrelevance_228: current time in Sydney asked; a local-time-by-city tool is available. REVIEW FLAG (fairness): get_local_time(location='Sydney') directly answers this, so a competent human would legitimately call it. BFCL nonetheless files it under irrelevance; kept as such to preserve the source label, but this item is effectively RELEVANT and should be reviewed. Marked Hard because the "no call" expectation is contradicted by the tool itself.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_local_time' directly answers 'current time in Sydney' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForTomatoCaloriesBFCL/irrelevance_222: CALORIES in a tomato asked; only a grocery-item-details lookup available. REVIEW FLAG (fairness): item_details at 'detailed' level could arguably surface nutrition info, and it requires a store_location the user never gave; BFCL labels it irrelevant (it is a grocery-store item lookup, not a nutrition-facts service). Kept as such; marked Medium as borderline.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 578 OUT641
✓	NoToolForWriteABookBFCL/irrelevance_217: "can you write a book" asked; only a recipe-CREATION tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN2 576 OUT204
Tool Calling - Irrelevance: Weather, Environment & Services 10 passed	100.0%	IN36 617 OUT5 315
✓	NoToolForBestCo2ReductionBFCL/irrelevance_108: the best way to reduce CO2 asked; only a numeric emission-reduction estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 660 OUT465
✓	NoToolForBestMonthToVisitBFCL/irrelevance_101: best month to visit Hawaii asked; only an average-monthly-temperature tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 658 OUT683
✓	NoToolForCactusNutrientsBFCL/irrelevance_109: a plant's nutrient needs asked; only a weekly watering-needs calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 665 OUT380
✓	NoToolForCurrentTimeWeatherBFCL/irrelevance_103: the current time asked; only a current-weather forecast tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 659 OUT319
✓	NoToolForGardenSpeciesCompositionBFCL/irrelevance_107: garden species composition asked; only a soil sand/silt/clay composition analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 662 OUT545
✓	NoToolForGmoYogaMatBFCL/irrelevance_111: a GMO yoga mat asked; only an eco-friendly product locator is available (GMO is not eco-friendly) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 661 OUT578
✓	NoToolForImpactfulCasesBFCL/irrelevance_94: a firm's impactful cases asked; only a single-case detail lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 669 OUT533
✓	NoToolForPaintingEiffelCaseResultBFCL/irrelevance_96: how long to paint the Eiffel Tower asked; only a law-case result lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 661 OUT423
✓	NoToolForPollinationFlowersBFCL/irrelevance_105: top pollination flower species asked; only a route elevation-profile plotter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 663 OUT861
✓	NoToolForTomatoFertilizerBFCL/irrelevance_106: best fertilizer for tomatoes asked; only a soil-type analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 659 OUT528
Tool Calling - Parallel Calls: Art, Music & Local Services 10 passed1 skipped	100.0%	IN75 758 OUT7 196
–	CalculateDrivingDistancesShortestRouteBFCL/parallel_38: two calc_distance calls — New York→Washington DC and Los Angeles→San Francisco. Tightened per methodology: the prompt explicitly enables "shortest route", so shortest_route == true is required (schema default is false).0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Skipped: Ambiguous scope (item flaw, not model behavior): the trailing 'with optional parameter shortest route enabled' can reasonably apply to only the second leg (its nearest clause) or to both; the BFCL ground truth requires it on both, but a competent human would not answer deterministically. Kept for traceability.	Skipped	IN0 OUT0
✓	CheckInstrumentAvailabilityInTwoCitiesBFCL/parallel_42: two check_instrument_availability calls — Yamaha P125 in Berlin and Madrid.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 484 OUT387
✓	FindMusicStoresInTwoCitiesBFCL/parallel_41: two find_music_instrument_store calls — San Francisco and Chicago; instruments {Yamaha acoustic guitar, Kawai piano} in any order and either casing.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 527 OUT531
✓	FindRockAndJazzConcertsInTwoCitiesBFCL/parallel_43: four concert_finder calls — {San Francisco, New York} x {rock, jazz} over the next month. time_period may be omitted (defaults to 30 days ≈ "next month"). Isolated from concert_find_nearby (parallel_44) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 826 OUT746
✓	GetExhibitionDetailsForTwoMuseumsBFCL/parallel_40: two museum_exhibition_detail calls — 'Wonder of Nature' at the Louvre and 'Age of Reptiles' at the British Museum. visitor_type {child, adult} (unordered). Tightened per methodology: the prompt explicitly asks for cost "for children and adult", so both visitor types are required (schema default is ['adult'] only).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 548 OUT1 218
✓	GetMostPlayedPopAndRockSongsBFCL/parallel_45: two musicCharts_getMostPlayed calls — most played Pop and Rock song in Australia. Optional duration omitted (ground truth accepts the schema default of 0).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 470 OUT630
✓	GetMuseumInfoForTwoCitiesBFCL/parallel_39: two museum_info_get_info calls — Washington D.C. and Paris; details {Opening hours, Adult tickets, Child tickets} in any order.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 500 OUT462
✓	GetRgbAndHexForThreeColorsBFCL/parallel_37: three color_converter_get_color_info calls — navy, purple, maroon; each with {RGB, HEX} (unordered).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 646 OUT617
✓	GetTeamRankingsInTwoLeaguesBFCL/parallel_47: two get_team_ranking calls — Barcelona in UEFA Champions League and Manchester United in La Liga.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 471 OUT451
✓	MixPaintColorsForTwoRequestsBFCL/parallel_36: two paint_color_mixture calls — watercolor magenta and acrylic navy.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 520 OUT1 477
✓	WinningPercentageForTwoTeamsTwoSeasonsBFCL/parallel_46: four calculate_winning_percentage calls — {Lakers, Bulls} x {2018, 2020}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 766 OUT677
Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 1) 13 passed	100.0%	IN111 888 OUT18 502
✓	AverageFourNumberSetsBFCL/parallel_132: four calculate_average calls — one per list of numbers (element order preserved).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 785 OUT730
✓	CalculateBmiFourPeopleMetricBFCL/parallel_134: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt states "All measurements are in the metric system", so unit == "metric" is required (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 803 OUT723
✓	CalculateCityDistancesInKilometersBFCL/parallel_135: three geo_distance_calculate calls — NY->LA->Miami->NY, in kilometers. Tightened: the tool defaults to miles but the prompt asks "in kilometers", so units == "kilometers" is REQUIRED (omission would return miles, contradicting the request — the ground truth's "omit" option is unsafe here).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 523 OUT2 270
✓	CalculateGpsDistancesAlongRouteBFCL/parallel_133: three calculate_distance calls over GPS coordinate pairs (Paris->Rome->Athens-> Giza), each in kilometers. Uses the coordinate-based tool (coord1/coord2), not the city-name distance tools that are also visible.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 827 OUT6 764
✓	CompareAverageTemperatureNyAndLaBFCL/parallel_128: two average_temperature calls — New York 7 days in Fahrenheit, Los Angeles 7 days in Celsius. Tightened: the prompt explicitly requests Fahrenheit for NY, so temp_unit == "Fahrenheit" is required (also the default); Celsius is non-default and required for LA.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 389 OUT555
✓	CreateTwoHistogramsWithFiveBinsBFCL/parallel_129: two create_histogram calls — one per data set, 5 bins each (data order preserved).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 822 OUT710
✓	EstimateElephantPopulationThreeScenariosBFCL/parallel_125: three elephant_population_estimate calls — current 500 at 2%, 1.5%, 2.5% over 10y.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 699 OUT2 429
✓	FindItalianRestaurantsInNyAndLaBFCL/parallel_130: two find_restaurants calls — New York and Los Angeles, Italian, 4 results each, dietary_requirements [vegan, gluten-free]. Distinct from restaurant_find_nearby (this tool carries food_type + number, which the prompt supplies, so it is the correct choice here).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 422 OUT651
✓	FindRestaurantsByDietInThreeCitiesBFCL/parallel_127: three restaurant_find_nearby calls — New York, Los Angeles, Chicago, each with dietary_preference [Vegan, Gluten-free, Dairy-free]. List order follows the prompt (matches ground truth).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 545 OUT518
✓	PredictEvolutionDarwinAndLamarckBFCL/parallel_126: two prediction_evolution calls — African Elephant over 5000y, Darwin then Lamarck. Tightened per methodology: the prompt explicitly says "using the Darwin model", so model == "Darwin" is required for the first call (ground truth also accepts omission via the 'Darwin' default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 433 OUT534
✓	ProjectPopulationGrowthJapanAndIndiaBFCL/parallel_124: four population_projections calls — Japan/10y and India/20y, each at the current (default) growth rate and at an explicit rate. "current growth rate" maps to the default (growth_rate omitted, accepts null or 0.01); the explicit 1.5%/2.1% rates are required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 706 OUT1 463
✓	RouteThreeLegsAvoidingTollsBFCL/parallel_131: three map_routing_fastest_route calls — SF->Palo Alto->San Jose->home, all avoiding tolls. Tightened: the prompt explicitly says "avoiding toll roads", so avoid_tolls == true is required for every leg (non-default value stated in the prompt).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 551 OUT677
✓	ShortestCityDistanceBusWithAndWithoutTransferBFCL/parallel_136: two city_distance_find_shortest calls — NY->LA by bus, first without transfer then allowing transfer. Tightened: the prompt explicitly says "using a bus", so transportation == "bus" is required for both (also the default). allow_transfer: false/omitted for the first, true for the second.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 383 OUT478
Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 2) 14 passed	100.0%	IN125 790 OUT11 282
✓	CalculateAreaOfThreeTrianglesBFCL/parallel_143: three calc_area_triangle calls — (10,5), (15,7), (20,10).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 958 OUT585
✓	CalculateBinomialProbabilityBasketballBFCL/parallel_161: three calculate_binomial_probability calls — basketball shooter at 60%: (10,7), (15,10), (20,15). probability_of_success == 0.6 is required (prompt states 60%). ISOLATED here (away from prob_dist.binomial in ToolCalling-Parallel-D3): both tools compute binomial probability, and this prompt does not name a function, so having both visible would make the choice genuinely ambiguous. Kept apart so exactly one correct tool is available.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 037 OUT2 423
✓	CalculateBmiInMetersForThreePeopleBFCL/parallel_138: three calculate_BMI calls (weight kg, height meters) — John, Sarah, Mike.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 019 OUT706
✓	CalculateCelestialDistancesInMilesBFCL/parallel_149: two calculate_distance calls between celestial bodies (Mars-Venus, Mars-Jupiter), in miles. Uses the body-name distance tool (body1/body2). Tightened: prompt asks "in miles", so unit == "miles" is required (tool default is kilometers, so omission would be wrong).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 764 OUT522
✓	CalculateClockAnglesAtThreeTimesBFCL/parallel_145: three calculate_clock_angle calls — 3:15, 8:20, 11:50. Tightened: the prompt asks for results "rounded to 2 decimal places", so round_to == 2 is required (also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 939 OUT894
✓	CalculateFactorialOfFourNumbersBFCL/parallel_144: four math_factorial calls — 5, 3, 4, 2.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 968 OUT388
✓	CalculateLightTravelTimeThreeLegsBFCL/parallel_147: three light_travel_time calls — 4.22, 6.1, 5.88 light years. Tightened: the prompt explicitly states the speed of light (299792458 m/s), so speed_of_light == 299792458 is required (also default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 025 OUT1 614
✓	CalculateSpeedForCarBicycleTrainBFCL/parallel_148: three calculate_speed calls — car 500m/25s in km/h, bicycle 1000m/200s in m/s, train 10000m/600s in km/h. Each output unit is explicitly named in the prompt, so to_unit is required.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 040 OUT646
✓	FetchAllDataFieldsForTwoEmployeesBFCL/parallel_139: two employee_fetch_data calls — Tech Solutions, employees 12345 and 67890, each fetching all four data fields (order follows the prompt / ground truth).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 872 OUT447
✓	FindDiCaprioDramaAndComedyMoviesBFCL/parallel_140: two imdb_find_movies_by_actor calls — Leonardo DiCaprio, Drama in 2010 and Comedy in 2012. Tightened: the prompt explicitly names each category, so category is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 760 OUT749
✓	GetImaxAndTwoDTheaterReleasesBFCL/parallel_141: two get_theater_movie_releases calls — New York IMAX over 7 days, Los Angeles 2D over 14 days. Tightened: the prompt explicitly names each format, so format is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 789 OUT442
✓	PlotTwoSineWavesBFCL/parallel_146: two plot_sine_wave calls with all five parameters supplied by the prompt.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 895 OUT390
✓	SortFourListsAscendingAndDescendingBFCL/parallel_137: eight array_sort calls — four lists, each sorted ascending AND descending (list element order preserved as given).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 844 OUT1 057
✓	UpdateTwoCustomersNameAndEmailBFCL/parallel_142: two update_user_info calls — users 12345 and 67890, both set name "John" and email "example@.com". update_info is a nested object. Tightened: the prompt names the 'CustomerInfo' database, so database == "CustomerInfo" is required (also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 880 OUT419
Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 3) 10 passed1 failed	90.9%	IN88 034 OUT26 746
✓	AreaUnderCurveForTwoPolynomialsBFCL/parallel_150: two mathematics_calculate_area_under_curve calls — polynomial [3,-2,1] over [-1,2] and [1,0,-1] over [0,3] (coefficient/limit order preserved).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 830 OUT1 081
✓	BinomialProbabilitiesThreeGamesBFCL/parallel_160: three prob_dist_binomial calls — (20,5), (50,15), (100,30), each with p == 0.3 (prompt states a 0.3 win probability). The prompt explicitly names 'prob_dist.binomial'; the colliding calculate_binomial_probability tool is kept in a separate script (ToolCalling-Parallel-D2).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 001 OUT19 592
✓	CalculateBmiSystemFourPeopleBFCL/parallel_154: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt says "use the metric system for all calculations", so system == "metric" is required (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 128 OUT868
✓	DiceRollProbabilitiesThreeScenariosBFCL/parallel_159: three probability_dice_roll calls — roll 4 three times and roll 2 twice on a six-sided die (die_sides default 6, accepts omission), then roll 7 twice on an eight-sided die (die_sides == 8 required, non-default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 958 OUT762
✓	GenerateNormalVariatesTwoConfigsBFCL/parallel_158: four random_normalvariate calls — two at (mu 5, sigma 2) and two at (mu 10, sigma 3). Duplicate calls: each configuration must appear exactly twice (multiplicity 2).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 143 OUT1 135
✓	PredictPricesForFourHousesBFCL/parallel_157: four predict_house_price calls — New York (3/2/1500), Los Angeles (4/3/2000), Chicago (2/1/1200), Miami (3/2/1800).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 252 OUT708
✓	RaiseTwoNumbersToPowersBFCL/parallel_152: two math_power calls — 3^5 and 2^3, no modulus (mod omitted / null).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 684 OUT365
✓	RunLinearRegressionWithAndWithoutStandardizationBFCL/parallel_155: two run_linear_regression calls — same predictors [Age, Income, Education] and target "Spending Score", first without standardization then with. standardize is explicitly requested (false then true), so it is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 896 OUT447
✓	TrainRandomForestFourConfigsBFCL/parallel_156: four random_forest_train calls on "data_random_forest" — (100,10), (200,20), (300,30), (400,40). The prompt explicitly names this function, disambiguating it from train_random_forest_classifier visible in this script.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 310 OUT648
✓	TrainRfClassifierTwoConfigsBFCL/parallel_153: two train_random_forest_classifier calls — dataset1 (depth 10, 100 trees) and dataset2 (depth 20, 200 trees). The prompt explicitly names this function, disambiguating it from the other random-forest training tool (random_forest_train) visible in this script.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 874 OUT426
×	TriangleAreasInDifferentUnitsBFCL/parallel_151: three geometry_area_triangle calls in different units — (15,20) square meters, (25,30) square feet, (35,40) square inches. First unit is the default (accepts omission / square meters); feet and inches are non-default and required.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: match0 == 1 Values: match0 = 0	Failed	IN7 958 OUT714
Tool Calling - Parallel Calls: Finance, Stats & Legal 13 passed	100.0%	IN110 764 OUT39 614
✓	BuildTwoDecisionTreeClassifiersBFCL/parallel_12: two model_DecisionTreeClassifier calls — gini/depth 5 and entropy/depth 10, both random_state 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 440 OUT353
✓	CalculateFutureValueForThreeTermsBFCL/parallel_18: three financials_calculate_future_value calls — years 1, 5, 10 (same 1000 @ 0.05).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 602 OUT756
✓	CalculatePresentValueForThreeTermsBFCL/parallel_14: three calculate_present_value calls — years 10, 20, 30 (same payment 1000, rate 0.05).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 599 OUT1 077
✓	CapitalGainsTaxForTwoStatesBFCL/parallel_15: two calculate_capital_gains_tax calls — CA (15000/25000) and FL (20000/50000).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 492 OUT4 043
✓	ComputePythagorasForTwoTrianglesBFCL/parallel_10: two math_pythagoras calls — sides (3,4) and (5,12).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 383 OUT363
✓	ConfidenceIntervalForTwoSampleSizesBFCL/parallel_13: two confidence_interval_calculate calls — sample sizes 50 and 150. Tightened per methodology: the prompt explicitly requests "95% confidence interval", so confidence_level == 0.95 is required (schema default is 0.9, but the prompt states the value).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 495 OUT1 318
✓	CourtCaseStatusAndTrialDatesBFCL/parallel_22: four court_info_get_case_status calls — cases XY1234 (LA) and GH5678 (Orange County), each queried for status (details may be omitted via the 'status' default) and for trial_date.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 803 OUT832
✓	GetStockPriceAndVolumeForThreeCompaniesBFCL/parallel_17: three get_stock_data calls — AAPL, GOOG/GOOGL, MSFT; each with data_points {price, volume} (unordered).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 534 OUT685
✓	LoanEligibilityForTwoInstitutionsBFCL/parallel_20: two loan_eligibility_check calls — HSBC (500k/100k) and Wells Fargo (700k/120k).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 507 OUT556
✓	MortgagePaymentForThreeLoanTermsBFCL/parallel_19: three calculate_mortgage_payment calls — loan terms 15, 20, 30 (400000 @ 0.04).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 627 OUT28 207
✓	PredictHousePriceInTwoCitiesBFCL/parallel_11: two ml_predict_house_price calls — New York 3000 sq ft, Los Angeles 4000 sq ft.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 414 OUT411
✓	ReturnOnInvestmentGainAndLossBFCL/parallel_16: two calculate_return_on_investment calls — gain 500 on 2000, loss -1000 on 5000.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 445 OUT424
✓	SearchMoneyLaunderingConvictionsBFCL/parallel_21: two law_crimes_search calls — money laundering in San Francisco 2019 and Texas 2018.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 423 OUT589
Tool Calling - Parallel Calls: Legal, History & Environment 12 passed1 failed1 skipped	92.3%	IN122 477 OUT12 349
✓	BookTwoConnectingFlightsBFCL/parallel_30: two book_flight calls — San Francisco→Tokyo (2022-05-03) and Tokyo→Sydney (2022-05-18).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 057 OUT284
✓	CalculateAlimonyForTwoDurationsBFCL/parallel_23: two alimony_calculator_ca_calculate calls — durations 10 and 20 years (10000/3000 income).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 120 OUT1 418
✓	CalculateWasteForFamilyAndBachelorBFCL/parallel_29: two waste_calculation_calculate calls — LA family (2 adults, 2 children) and New York bachelor (1 single). The population argument is a nested object.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 070 OUT1 180
✓	FetchTwoHistoricalFactsBFCL/parallel_31: two history_fact_fetch calls — Treaty of Paris and Magna Carta. Optional depth/year omitted (ground truth accepts the schema defaults).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 976 OUT917
✓	FindClassicalConcertsWithParkingBFCL/parallel_44: two concert_find_nearby calls — classical concerts near Berlin and Paris next Friday. Tightened per methodology: the prompt explicitly asks for concerts "with available parking", so amenities == ['Parking'] is required (this also matches the schema default, but the prompt states it). Isolated from concert_finder (parallel_43) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 050 OUT1 065
–	FindCourtCasesByNumberAndTypeBFCL/parallel_26: four court_case_find calls — the cross product of case numbers {67813, 71249} and types {Civil, Criminal}; case_number is a single-element array; Civil may be omitted via its default. FLAGGED as borderline (see report): the prompt "case numbers '67813','71249' for type 'Civil' and 'Criminal'" is ambiguous about the pairing; BFCL ground truth expects all four combinations, kept faithful.0 AI queries 0 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL Skipped: Ambiguous call decomposition (item flaw, not model behavior): 'case numbers 67813, 71249 for type Civil and Criminal' has no single deterministic reading - two calls (both numbers per type, case_number as a list) is as valid as the BFCL 2x2 = four single-number calls; a competent human would not answer deterministically. Kept for traceability.	Skipped	IN0 OUT0
✓	FindNatureReservesNearTwoCitiesBFCL/parallel_27: two nature_reserve_find_nearby calls — Berkeley (10km, picnic tables + restrooms) and Tokyo (5km, playgrounds + biking trails). Amenity list order is unordered.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 058 OUT514
✓	GetCaseDetailsWithHistoryAndLitigantsBFCL/parallel_24: two law_case_get_details calls — cases 28473 and 64725. Tightened per methodology: the prompt explicitly asks for "their history and details of litigants", so include_history == true and include_litigants == true are required (schema defaults are false).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 040 OUT608
✓	GetEventsForTwoPresidenciesBFCL/parallel_32: two us_history_events_by_presidency calls — Abraham Lincoln and George Washington. Optional start_year/end_year omitted (ground truth accepts the schema defaults).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 971 OUT812
✓	GetPresidentsAndVicePresidentsByYearBFCL/parallel_33: four get_president_and_vp calls — president for 1980 and 2016, vice president for 1975 and 2011.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 326 OUT981
✓	GetReligionInfoForTwoEmpiresBFCL/parallel_35: two ancient_empires_get_religion_info calls — Persian Empire and Mauryan Empire. Tightened per methodology: the prompt explicitly asks for "religious history and influences", so include_influences == true is required (schema default is false).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 011 OUT777
✓	GetTemperatureForTwoCitiesBFCL/parallel_28: two get_current_and_future_temperature calls — Seattle and Los Angeles, next 3 hours.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 983 OUT320
×	LookupLawsuitsForTwoYearsBFCL/parallel_25: two lawsuit_lookup calls — Dara Inc for years 2019 and 2018.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 2	Failed	IN13 761 OUT1 507
✓	TrackChristianityInTwoRegionsBFCL/parallel_34: two religion_history_track calls — Christianity in Egypt and Turkey, 100–1500 A.D.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 054 OUT1 966
Tool Calling - Parallel Calls: Markets & Environment 13 passed	100.0%	IN114 708 OUT11 585
✓	CalculateCagrForTwoInvestmentsBFCL/parallel_176: two calculate_cagr calls — (5000,7000,5) and (8000,12000,3).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 665 OUT769
✓	CalculateEmissionSavingsForSolarAndWindBFCL/parallel_186: two calculate_emission_savings calls — solar(12 months) and wind(8 months), Midwest region. Prompt states the Midwest region explicitly -> region required (accept "Midwest"/"Midwest region").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 558 OUT549
✓	CompareFutureValueWithAndWithoutContributionsBFCL/parallel_182: two finance_calculate_future_value calls — (5000,0.07,10, no contribution) and (3000,0.06,10, contribution 200). First scenario states "no additional contributions" -> null or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 725 OUT1 383
✓	ComputeEuclideanNormForThreePointsBFCL/parallel_183: three math_hypot calls — (5,7), (10,15), (20,25). z omitted or 0 (2D points).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 761 OUT1 713
✓	EstimatePopulationsForTwoSpeciesAcrossTwoYearsBFCL/parallel_185: four estimate_population calls — Bengal Tigers/India and African Elephants/Kenya for 2021 and for 2023. NOTE: upstream ground truth's third entry lists year "" (omitted -> default 2020), which is an upstream error: the prompt explicitly asks for 2023 for "both these species". We tighten both 2023 estimates to year == 2023 (fair deterministic answer).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 904 OUT623
✓	FindRootsForTwoQuadraticEquationsBFCL/parallel_184: two algebra_quadratic_roots calls — (3,7,2) and (5,-4,1).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 678 OUT917
✓	GetClosingPricesForTwoCompaniesOnTwoDatesBFCL/parallel_178: four get_stock_price (by-date variant) calls — the 2x2 cross product of {Microsoft, Apple} x {2022-01-01, 2022-02-01} on NASDAQ. Upstream ground truth's first entry loosely lists company_name as [Microsoft, Apple]; the deterministic human answer is the four distinct (company, date) combinations, which we require. exchange NASDAQ (default) may be omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 894 OUT748
✓	GetDetailedAirQualityForThreeDaysBFCL/parallel_187: three get_air_quality calls — New York City, detail true, for 2023-05-05, 05-04, 05-03. Prompt asks for PM2.5/PM10/ozone/sources -> detail == true; "today/yesterday/day before" with today = May 5, 2023.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 764 OUT741
✓	GetFourPriceTypesForTwoCompaniesBFCL/parallel_180: eight stock_price calls — {Microsoft, Apple} x {Open, Close, High, Low}, days 30. Prompt explicitly lists all four data types, so data_type must be set explicitly (required) to distinguish the eight calls; omission (default 'Close') cannot represent the four requested types.1 AI query 8 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL	Passed	IN9 455 OUT975
✓	GetPricePerOunceForFourMetalsBFCL/parallel_177: four get_metal_price calls — gold, silver, platinum, palladium, measure 'ounce'. Prompt asks "price per ounce" -> measure == "ounce" required.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 785 OUT592
✓	GetStockPricesForFourCompaniesAndDurationsBFCL/parallel_181: four get_stock_prices calls — Apple(1 week), Microsoft(2 weeks), Amazon(3 weeks), Tesla(1 month). Each 'companies' argument is a single-element list.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 834 OUT455
✓	GetTrafficInfoForThreeLegsWithModesBFCL/parallel_188: three get_traffic_info calls — NY->LA (driving), LA->SF (bicycling), SF->NY (transit). First leg explicitly says "driving" -> mode == "driving" required (tightened; also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 759 OUT471
✓	ProjectFutureValueForThreeStocksBFCL/parallel_175: three portfolio_future_value calls — AAPL(5000,0.07,5), MSFT(8000,0.06,7), AMZN(10000,0.08,10).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 926 OUT1 649
Tool Calling - Parallel Calls: Mathematics & Physics (set 1) 12 passed1 failed	92.3%	IN104 061 OUT9 113
✓	AreaOfTwoRightTrianglesInMetersBFCL/parallel_68: two calculate_area calls — right triangles base/height 12x15 and 18x24 in meters. unit required (tool default is 'cm'; the prompt states meters/square meters).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 990 OUT501
✓	CheckPlayerStatusAcrossThreeSeasonsBFCL/parallel_49: three player_status_check calls — player 3142 in RocketLeague for seasons 2017/2018/2019. Seasons are explicitly given, so each is required (default 2023 not applicable).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 071 OUT644
✓	CircumferenceOfFourCirclesInCmBFCL/parallel_65: four calculate_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt gives every radius in cm (e.g. "5cm") and the tool default unit is 'm', so a correct call must pass unit == 'cm'/'centimeter' (ground truth requires it on the first call and merely allows omission on the rest; we require it consistently since it is stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 213 OUT564
×	ConvertEuroAndPoundsToDollarsBFCL/parallel_59: two get_conversion_cost calls — 15000 Euro->dollars and 200 pounds->dollars.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: match1 == 1 Values: match1 = 0	Failed	IN7 901 OUT1 156
✓	FinalVelocityForFourDropsBFCL/parallel_81: four calculate_final_velocity calls — heights 10/20/15/25, initial_velocity 0 (stated), gravity 9.81 (default, may be omitted).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 398 OUT1 165
✓	FindItalianAndJapaneseRestaurantsBFCL/parallel_52: two restaurant_finder calls — Italian in New York (preferences ['Vegetarian']) and Japanese in Los Angeles (delivery; ['Delivery'] or omitted since default is ['Delivery']).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 896 OUT687
✓	FindRootsOfTwoQuadraticsAlgebraBFCL/parallel_62: two algebra_quadratic_roots calls — 3x^2+4x+2 and 5x^2-7x+3.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 063 OUT789
✓	FindSoupAndVeganSaladRecipesBFCL/parallel_51: two recipe_search_find calls — Chicken Noodle Soup (diet omitted/default 'Keto') and a Vegan Salad (diet == 'Vegan', dish name accepts several phrasings).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 831 OUT529
✓	GcdOfJohnAndMaryNumbersBFCL/parallel_80: two math_gcd calls — gcd(36,48) and gcd(60,96). Same tool as parallel_76.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 946 OUT427
✓	GcdOfTwoPairsBFCL/parallel_76: two math_gcd calls — gcd(45,60) and gcd(81,27).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 878 OUT313
✓	GetPokemonMovesAndCheckSolarBeamBFCL/parallel_48: two PokemonGO_get_moves calls — Pikachu's move list (move omitted/default 'Run') and a check whether Bulbasaur can learn 'Solar Beam'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 868 OUT820
✓	PrimeFactorsFormattedAndArrayBFCL/parallel_74: four get_prime_factors calls — numbers 4567 and 7890, each formatted true and false (user wants "a formatted string as well as an array").1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 160 OUT1 198
✓	SaveGameProgressEasyAndHardBFCL/parallel_50: two game_save_progress calls — stage 7 easy and stage 3 hard; level omitted/default 'user'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 846 OUT320
Tool Calling - Parallel Calls: Mathematics & Physics (set 2) 13 passed	100.0%	IN107 949 OUT11 555
✓	AreaOfThreeCirclesInMetersBFCL/parallel_66: three geometry_area_circle calls — radii 5/10/15 in meters (default is meters, so units may be omitted).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 335 OUT640
✓	BookTwoHotelRoomsBFCL/parallel_57: two hotel_booking_book calls — Sheraton (NY, 2 adults/1 child) and Marriott (LA, 1 adult/2 children).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 383 OUT468
✓	CheckAvailabilityForTwoHotelsBFCL/parallel_56: two hotel_booking_check_availability calls — Queens Hotel (Berlin) and Royal Hotel (Paris), each with check-in/out dates (several date formats accepted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 315 OUT584
✓	CircumferenceOfFourCirclesInMetersBFCL/parallel_70: four geometry_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt asks for output in meters and gives radii in m, but the tool default unit is 'cm', so a correct call must pass units == 'm'/'meters' (ground truth requires it on the first call and only allows omission on the rest; we require it consistently since it is explicitly stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 515 OUT685
✓	DerivativeOfPolynomialAndItsDerivativeBFCL/parallel_71: two calculate_derivative calls — 3x^3-2x^2+5x-7 at x=4, then its derivative 9x^2-4x+5 at x=2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 230 OUT1 125
✓	FinalVelocityForThreeDragRaceRoundsBFCL/parallel_83: three final_velocity calls — (v0=0,a=5,t=10), (v0=10,a=7,t=8), (v0=20,a=4,t=12).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 512 OUT496
✓	FindGroceryStoresByCriteriaBFCL/parallel_55: two grocery_store_find_by_criteria calls — NYC ['24 hours'] and San Diego ['Home Delivery'].1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 160 OUT484
✓	GetTwoCookingRecipesBFCL/parallel_53: two get_cooking_recipe calls — Lasagne Bolognese for 4 and Caesar Salad for 2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 166 OUT1 075
✓	GetTwoExchangeRatesBFCL/parallel_58: two get_exchange_rate calls — USD->AUD and USD->CAD.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 130 OUT323
✓	HighestCommonFactorOfFourPairsBFCL/parallel_77: four math_hcf calls — (45,60), (90,120), (36,48), (72,96). FLAGGED (see final report): the prompt is phrased as chaining ("use that result to find the HCF with...") but the ground truth treats it as four INDEPENDENT parallel calls on the literal pairs; a human might interpret it as sequential composition. Kept BFCL-faithful (four independent literal-pair calls) since parallel semantics preclude feeding one call's result into another.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 549 OUT3 495
✓	OrderSameItemsFromTwoWholeFoodsLocationsBFCL/parallel_54: two whole_foods_order calls — same two items (order-independent) at downtown and uptown, size large.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 212 OUT370
✓	PrimeFactorizeAsDictionaryBFCL/parallel_79: two prime_factorize calls — 4567 and 7890, return_type 'dictionary' (explicitly requested).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 172 OUT1 192
✓	SolveTwoQuadraticEquationsBFCL/parallel_63: two solve_quadratic_equation calls — (5,6,1) and (3,2,1). Prompt names this function explicitly.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 270 OUT618
Tool Calling - Parallel Calls: Mathematics & Physics (set 3) 12 passed	100.0%	IN98 273 OUT8 588
✓	AreaOfTwoCirclesInMetersBFCL/parallel_67: two geometry_calculate_area_circle calls — radii 5 and 10 in meters. unit required (tool default is 'cm'; the prompt states meters).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 019 OUT577
✓	AreaOfTwoTrianglesBFCL/parallel_69: two calculate_triangle_area calls — base/height 10x5 and 8x6.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 079 OUT498
✓	DerivativeOfTwoFunctionsAtValuesBFCL/parallel_73: two calculus_derivative calls — 3x^2+2x-1 at 5 (variable x, default) and 4y^3-3y^2+2y-1 at 3 (variable y).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 124 OUT630
✓	DisplacementForThreeCarsBFCL/parallel_84: three calculate_displacement calls — (15,7,accel 3.5), (20,10,accel 2.0), (25,8,no acceleration). The third car travels "without any acceleration" so acceleration == 0 (also the default; omission accepted).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 340 OUT813
✓	EuclideanNormForThreePointsBFCL/parallel_61: three math_hypot calls — (3,4) and (6,8) with z omitted/0, and (9,12,15).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 254 OUT586
✓	FactorialOfThreeNumbersBFCL/parallel_60: three math_factorial calls — 5, 7, 9.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 081 OUT401
✓	FinalSpeedForTwoFreeFallObjectsBFCL/parallel_85: two calculate_final_speed calls — initial_speed 0 for 10 s and initial_speed 5 for 7 s; gravity -9.81 (default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 147 OUT1 098
✓	GcdViaNumberTheoryBFCL/parallel_78: two number_theory_gcd calls — (45,60) and (81,63). Prompt names this function explicitly.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 022 OUT410
✓	IntegrateTwoFunctionsWithTwoMethodsBFCL/parallel_72: four integrate calls — x3 on [2,5] and 2x2+3x-1 on [-1,3], each with trapezoid (default, may be omitted) and simpson (explicit).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 468 OUT1 439
✓	PrimeFactorsOfThreeNumbersBFCL/parallel_75: three number_analysis_prime_factors calls — 45, 100, 150.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 087 OUT449
✓	SolveFourQuadraticsWithRootTypeBFCL/parallel_64: four solve_quadratic calls — (2,5,3) all roots, (1,-3,2) real, (4,-7,3) all roots, (1,2,1) real. 'all' is the default so it may be omitted; 'real' is explicitly requested and required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 560 OUT1 040
✓	VelocityForTwoCyclingDaysBFCL/parallel_82: two calculate_velocity calls — 120 km in 5 h and 150 km in 6 h; unit km/h (default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 092 OUT647
Tool Calling - Parallel Calls: Mixed Domains 8 passed2 failed	80.0%	IN73 380 OUT5 603
✓	CalculateBmiForTwoPeopleBFCL/parallel_4: two calculate_bmi calls — 6 ft / 80 kg and 5.6 ft / 60 kg.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 278 OUT491
✓	CalculateEmForceForTwoTimeIntervalsBFCL/parallel_1: two calculate_em_force calls — same field/area, d_time 4 then 10.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 299 OUT524
✓	CalculateFactorialForThreeNumbersBFCL/parallel_7: three math_factorial calls — 5, 10, 15.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 338 OUT385
✓	CalculateResistanceForCopperAndAluminumBFCL/parallel_2: two calculate_resistance calls — copper then aluminum, same length/area. Tightened per methodology: the prompt explicitly names "resistivity of copper", so the first call must pass resistivity == "copper" (ground truth also accepts omission via the 'copper' default, but an explicitly-stated value must be supplied). Aluminum is non-default → required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 286 OUT702
✓	CalculateSalesTaxForThreeCitiesBFCL/parallel_6: three calculate_sales_tax calls — Chicago/IL $30.45, Sacramento/CA $52.33, Portland/OR $11.23.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 470 OUT821
✓	FetchPopulationForCitiesStateAndCountryBFCL/parallel_8: four database_us_census_get_population calls — NYC (city), LA (city), Alaska (state), USA (country); year defaults to 2000 (may be omitted). Requires area-type inference.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 554 OUT567
×	FindMovieShowingsForTwoMoviesBFCL/parallel_9: two find_movie_showing calls near San Diego — Tenet at 5 pm and No Time To Die at 7:30 pm.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 1 expectedCount = 2	Failed	IN7 160 OUT792
×	GetSequenceAnd3DForThreeProteinsBFCL/parallel_3: three protein_info_get_sequence_and_3D calls — HbA1c, normal + rat hemoglobin. Tightened per methodology: the prompt explicitly asks for "their 3D models", so model_3d == true is required (ground truth also accepts omission via the default of true).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: match0 == 1 Values: match0 = 0	Failed	IN7 418 OUT495
✓	ListShowRatingsOnNetflixAndHuluBFCL/parallel_5: two streaming_services calls — Netflix['Friends'] and Hulu['The Office','Stranger Things'] (Hulu show order unordered), both sorted by rating. Must group shows per service.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 315 OUT535
✓	PlayTwoArtistsForGivenDurationsBFCL/parallel_0: two spotify_play calls — Taylor Swift for 20 min and Maroon 5 for 15 min.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 262 OUT291
Tool Calling - Parallel Calls: Physical & Life Sciences (set 1) 11 passed1 failed	91.7%	IN102 115 OUT32 256
✓	ElectricFieldForTwoChargesBFCL/parallel_94: two calculate_electric_field calls — charge 5/distance 2 and charge 3/distance 4. "Conducted in a vacuum" = the default permitivity → may be omitted or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 410 OUT727
✓	ElectromagneticForceInTwoMediaBFCL/parallel_96: two electromagnetic_force calls — same charges 5,10 and distance 2, differing by medium_permittivity. Vacuum call (8.854e-12) equals the default → may be omitted or 8.854e-12; the second medium (5e-12) is non-default → required.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 595 OUT3 786
✓	EstimateTravelTimeForThreeRoutesBFCL/parallel_93: three route_estimate_time calls — (1) NYC->Miami via Philadelphia/Washington D.C./Atlanta, (2) NYC->Miami skipping Atlanta (via Philadelphia/Washington D.C.), (3) Philadelphia-> Miami via Washington D.C. Stop orderings from ground truth accepted. FLAG parallel_93: BORDERLINE. Ground truth allows many stop-list variants, some including the start city itself as a stop; a competent human's stop set is not fully deterministic. Kept BFCL-faithful by accepting the natural stop orderings (start city excluded from stops).1 AI query 3 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL	Passed	IN8 558 OUT889
✓	FinalSpeedForTwoDroppedObjectsBFCL/parallel_88: two calculate_final_speed calls — v0 0/height 10 and v0 5/height 20. gravity stated as 9.8 (the default) so it may be omitted or 9.8 per ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 416 OUT522
✓	FinalVelocityForTwoCarsBFCL/parallel_87: two calculate_final_velocity calls — Car A (v0 0, a 6, t 10), Car B (v0 20, a 4, t 15).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 433 OUT499
✓	FinalVelocityFromDistanceForTwoObjectsBFCL/parallel_86: two kinematics_final_velocity_from_distance calls — accel 5/dist 100 and accel 10/dist 200. "Both objects start from rest" → initial_velocity may be omitted or 0 (default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 441 OUT634
✓	FindVeganRestaurantsInThreeCitiesBFCL/parallel_91: three vegan_restaurant_find_nearby calls — LA, SF, Seattle, each with operating_hours 22 (prompt: "open until at least 22:00", a non-default value → required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 485 OUT546
✓	GenerateFourTravelItinerariesBFCL/parallel_90: four travel_itinerary_generator calls — Tokyo/7/200/urban, Paris/10/150/history, Sydney/5/100/nature, Rome/12/180/culture. Tokyo's "urban" equals the default → omit or "urban".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 787 OUT675
×	GetDirectionsForRoundTripBFCL/parallel_89: three get_directions calls — SF->Palo Alto (fastest), Palo Alto->Golden Gate Bridge (scenic), Golden Gate Bridge->SF (fastest). route_type explicitly stated each time.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: match0 == 1 Values: match0 = 0	Failed	IN8 524 OUT2 746
✓	MagneticFieldForTwoLoopsBFCL/parallel_95: two calculate_magnetic_field calls — current 10/radius 0.5 and current 15/radius 1.0. "Same as in free space" = default permeability → may be omitted or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 516 OUT935
✓	ResonantFrequencyForTwoCircuitsBFCL/parallel_97: two calculate_resonant_frequency calls — L 0.005/C 1e-07/round 3 and L 0.007/C 2e-07/round 4. Rounding is stated explicitly → required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 480 OUT18 831
✓	ShortestDrivingDistanceRoundTripBFCL/parallel_92: three get_shortest_driving_distance calls — NYC->LA, LA->Miami, Miami->NYC, all in miles (prompt explicitly "in miles" → non-default unit required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 470 OUT1 466
Tool Calling - Parallel Calls: Physical & Life Sciences (set 2) 13 passed	100.0%	IN111 803 OUT16 741
✓	AbsolutePressureAtTwoAltitudesBFCL/parallel_102: two calc_absolute_pressure calls — gauge 2.5 with atm 1 (default; may be omitted) and gauge 2.5 with atm 0.85 (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 519 OUT1 408
✓	BoilingMeltingPointsAcrossSeaLevelsBFCL/parallel_100: four get_boiling_melting_points calls — water@0, iron@1000, water@1000, iron@0. FLAG parallel_100: BORDERLINE. The prompt ("water and iron at sea levels of 0 and 1000 meters respectively") reads most naturally as TWO calls (water@0, iron@1000), but the BFCL ground truth is the full 2x2 cross product (FOUR calls). Kept BFCL-faithful to the ground truth; a competent human could reasonably answer with two calls, so this item's expected call count is not fully deterministic.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 405 OUT1 793
✓	CellInfoDetailedAndBriefBFCL/parallel_108: two biology_get_cell_info calls — neuron with detailed=true (prompt: "detailed description"), muscle with detailed=false ("less detailed", the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 408 OUT2 368
✓	DensityForTwoSubstancesBFCL/parallel_101: two calculate_density calls (mass/volume schema) — 10 kg / 2 m³ and 15 kg / 3 m³. unit "kg/m³" equals the default → may be omitted or "kg/m³".1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 536 OUT451
✓	ElectricFieldStrengthAtSeveralDistancesBFCL/parallel_98: four calculate_electric_field_strength calls — charge 2, distances 0.5/1/2 in vacuum (default medium), plus distance 1 in air. The distance-1 vacuum and distance-1 air calls are distinguished by medium. Vacuum equals the default → may be omitted or "vacuum"; "air" is required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 831 OUT1 098
✓	EntropyChangeForTwoConditionsBFCL/parallel_103: two entropy_change_calculate calls — substance A, mass 2, temps 25->75 and 10->50. pressure 1 equals the default → may be omitted or 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 638 OUT879
✓	FetchDnaSequenceThreeWaysBFCL/parallel_106: three fetch_DNA_sequence calls for id XYZ123 — default fasta/0, then genbank format, then 500 bp upstream. Defaults (fasta, upstream 0) may be omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 613 OUT414
✓	FunctionLookupForTwoMoleculesBFCL/parallel_110: two cell_biology_function_lookup calls — ATP in mitochondria and DNA in nucleus, both with specific_function=true (prompt asks for a specific function within the organelle).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 469 OUT630
✓	HeatCapacityForThreeConditionsBFCL/parallel_105: three calc_heat_capacity calls — 300K/2.5m³, 350K/2.5m³, 300K/1.5m³, all for air. "air" equals the default gas → may be omitted or "air".1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 717 OUT2 973
✓	MolecularWeightForTwoCompoundsBFCL/parallel_111: two calculate_molecular_weight calls — Glucose (C6H12O6) and Sucrose (C12H22O11), both in grams/mole (prompt explicitly requests that unit; ground truth also accepts "g/mol").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 470 OUT624
✓	PhaseChangeEnergyForWaterBFCL/parallel_99: four thermo_calculate_energy calls — 500 g water, phase melting/freezing/ vaporization/condensation. Substance "water" equals the default → may be omitted or "water".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 902 OUT1 516
✓	ProteinSequencesForTwoGenesTwoSpeciesBFCL/parallel_107: four get_protein_sequence calls — BRCA1 and BRCA2 in Homo sapiens (default species, may be omitted) and in Pan troglodytes (non-default → required).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 732 OUT1 021
✓	ProteinsInThreeCompartmentsBFCL/parallel_109: three cellbio_get_proteins calls — nucleus/mitochondria/cytoplasm, each with include_description=true (prompt asks for "a brief description of each protein").1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 563 OUT1 566
Tool Calling - Parallel Calls: Physical & Life Sciences (set 3) 13 passed	100.0%	IN118 188 OUT12 251
✓	AnalyzeDnaForThreeMutationTypesBFCL/parallel_114: six analyze_dna_sequence calls — two sequences ("AGCTTAGCTA" and "AGCTTAGGCTA") against reference "AGCTTAGCTA", each checked for insertion (default, may be omitted), deletion, and substitution mutations.1 AI query 6 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL	Passed	IN13 733 OUT2 208
✓	DiabetesPredictionForFourPeopleBFCL/parallel_113: four diabetes_prediction calls for four people with distinct weight/height/activity.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 962 OUT670
✓	EntropyChangeIsothermalAndNotBFCL/parallel_104: two calculate_entropy_change calls — same 300->350 K and heat_capacity 4.18, once isothermal (the default true, may be omitted) and once not isothermal (false, required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 651 OUT1 361
✓	ForestGrowthForTwoRegionsBFCL/parallel_120: two forest_growth_forecast calls — Amazon Rainforest over 10 years and Boreal Forests of Canada over 20 years, excluding human impact (false = the default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 515 OUT525
✓	GenerateFourDnaSequencesBFCL/parallel_123: four generate_DNA_sequence calls — length 500 with a single-nucleotide preference of A, T, C, G respectively.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 822 OUT577
✓	GeneticSimilarityInTwoFormatsBFCL/parallel_115: four genetics_calculate_similarity calls — human vs chimpanzee and human vs gorilla, each in percentage (default, may be omitted) and fraction. species1 accepts "human"/"Human".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 843 OUT1 432
✓	GenotypeFrequenciesForThreeGenotypesBFCL/parallel_116: three calculate_genotype_frequency calls — allele frequency 0.7, genotypes AA/Aa/aa.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 720 OUT906
✓	IdentifyFourBirdsBFCL/parallel_119: four identify_bird calls — small blue/forest (default size, may be omitted), large black/lake, medium brown/desert, large green/tropical rainforest.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 943 OUT1 088
✓	MutationTypeForTwoSnpsBFCL/parallel_112: two mutation_type_find calls — rs123456 in Homo sapiens (default species, may be omitted) and rs7891011 in Canis lupus familiaris (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 537 OUT590
✓	PopulationDensityForChinaTwoYearsBFCL/parallel_117: two calculate_density calls (country/population schema) — China in 2000 (pop 1.267 billion) and 2010 (pop 1.341 billion), land area 9.597 million km² both times.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 600 OUT1 087
✓	PrecipitationStatsForThreeTimeFramesBFCL/parallel_118: three ecology_data_precipitation_stats calls — Amazon rainforest for six_months, year, five_years.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 610 OUT533
✓	TurtlePopulationForTwoYearsBFCL/parallel_121: two ecology_get_turtle_population calls — Galapagos Islands in 2015 and 2020, both with species info requested (species=true; years stated explicitly → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 530 OUT488
✓	VehicleEmissionsForThreeVehiclesBFCL/parallel_122: three calculate_vehicle_emission calls — 15000 miles each: gas with default factor 1.4 (may be omitted), diesel with factor 2.7, EV with factor 0.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 722 OUT786
Tool Calling - Parallel Calls: Services & Lookups 11 passed1 failed	91.7%	IN98 389 OUT6 251
✓	CalculateShortestAndScenicDistancesForTwoLegsBFCL/parallel_190: four calculate_shortest_distance calls — {NYC->LA, LA->Miami} x {Shortest, Scenic}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 368 OUT838
✓	FetchNewsForTwoTopicsInEuropeBFCL/parallel_192: two get_news calls — Climate Change and Artificial Intelligence, 5 articles each, region Europe. Region "Europe" explicitly requested -> required (accept "Europe"/"EU").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 049 OUT273
✓	FindLibrariesWithFacilitiesInThreeCitiesBFCL/parallel_191: three public_library_find_nearby calls — NY[Reading Room,Fiction], LA[Wi-Fi,Children Section], Chicago[Cafe,Reading Room]. Facilities compared as an unordered pair.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 267 OUT550
✓	FindParksWithAmenitiesInThreeCitiesBFCL/parallel_189: three parks_find_nearby calls — NY[Tennis Court,Picnic Area], LA[Playground,Running Track], Chicago[Tennis Court,Playground]. Amenities compared as an unordered pair.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 248 OUT554
×	FindUpcomingEventsInThreeCitiesBFCL/parallel_194: three event_finder_find_upcoming calls — jazz/LA(14d), rock/Chicago(10d), classical/Boston(7d). All day windows explicitly stated -> days_ahead required (7 for Boston, which also equals the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: match0 == 1 Values: match0 = 0	Failed	IN8 262 OUT570
✓	GetCurrentWeatherForFourCitiesBFCL/parallel_199: four get_current_weather calls — New York, Los Angeles, London, Tokyo, with both temperature and humidity included. Prompt explicitly asks to include both -> flags == true (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 343 OUT611
✓	GetDetailedBriefsForTwoMoviesBFCL/parallel_195: two movie_details_brief calls — Inception and The Dark Knight, extra_info true. Prompt asks for Director/Cast/Awards -> extra_info == true required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 981 OUT259
✓	GetLawsuitDetailsForTwoCasesBFCL/parallel_196: two get_lawsuit_details calls — case 12345 (NY Supreme Court, with verdict) and case 67890 (LA Superior Court, no verdict). "would like the verdict" -> true; "do not need" -> null/false.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 168 OUT704
✓	GetLawsuitInfoForTwoCasesByYearBFCL/parallel_197: two lawsuit_info calls — 12345ABC (2018, New York) and 67890XYZ (2019, California). Years are explicitly stated -> required. First location "New York" may be omitted (matches the default).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 167 OUT468
✓	GetStockPricesForTwoCompaniesOverDaysBFCL/parallel_179: two get_stock_price (by-days variant) calls — Apple on NASDAQ for 10 days, Microsoft on NYSE for 15 days. Both exchanges are explicitly named -> required (NASDAQ / NYSE). The tool's 'company' field is a ticker symbol; company names are accepted as synonyms.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 050 OUT526
✓	SearchLawsuitsForTwoEntitiesInCaliforniaBFCL/parallel_198: two lawsuit_search calls — Google/Santa Clara and Facebook/San Mateo, state California. Prompt states "both in the state of California" -> state required (accept "California"/"CA").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 087 OUT372
✓	SendTwoEmailsWithCcAndBccBFCL/parallel_193: two send_email calls — Project Update to john.doe (cc manager, bcc hr) and Meeting Reminder to jane.doe (cc assistant, bcc hr). Bodies matched exactly.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 399 OUT526
Tool Calling - Parallel Calls: Statistics & Finance 13 passed	100.0%	IN117 775 OUT20 813
✓	CalculateBinomialProbabilitiesForBasketballShotsBFCL/parallel_163: three calc_binomial_prob calls — (10,6,0.6), (10,6,0.5), (15,6,0.5).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 157 OUT1 859
✓	CalculateCompoundInterestForThreePeriodsBFCL/parallel_170: three calculate_compound_interest calls — principal 5000, rate 0.025, time 2/3/5. Prompt states "compounded quarterly, 4 times in a year" -> n == 4 required (tightened; also ground truth).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 164 OUT4 818
✓	CalculateDiscountedCashFlowForTwoBondsBFCL/parallel_169: two calculate_discounted_cash_flow calls — (50,5,0.05) and (60,7,0.04). Prompt explicitly states "face value of the bond is $1000" for both -> face_value == 1000 required (ground truth also accepts omission via the default, but an explicitly-stated value must be supplied).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 012 OUT2 283
✓	CalculateEventProbabilitiesForCardDeckBFCL/parallel_162: three probability_of_event calls — Ace 4/52 as decimal, heart 13/52 as decimal, red card 26/52 as ratio. format_as_ratio: "as a decimal" -> false or omitted; "as a ratio" -> true (required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 124 OUT669
✓	CalculateQuarterlyDividendForThreeScenariosBFCL/parallel_168: three finance_calculate_quarterly_dividend_per_share calls — (5M,2M), (6M,2M), (6M,2.5M).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 288 OUT1 349
✓	CalculateReturnOnEquityForTwoCompaniesBFCL/parallel_171: two calculate_return_on_equity calls — (1M,5M,200k) and (2M,10M, no dividends). Second company "did not pay any dividends" -> dividends_paid null or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 955 OUT1 932
✓	CalculateRoiForThreeStocksBFCL/parallel_174: three calculate_return_on_investment calls — (150,180,20), (200,210,30), (250,300,40). Dividends are explicitly stated per stock -> required.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 096 OUT754
✓	ComputeRSquaredForTwoRegressionModelsBFCL/parallel_167: two linear_regression_get_r_squared calls on the same CSV — swap education/credit_score as dependent variable. Independent-variable order follows the prompt/ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 930 OUT687
✓	PredictFutureValueForTwoInvestmentsBFCL/parallel_172: two finance_predict_future_value calls — (5000,0.05,10) and (7000,0.04,15). Prompt states "compound interest annually" -> compounding_periods_per_year == 1 required (tightened; also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 940 OUT1 260
✓	PredictHousePricesForThreeHousesBFCL/parallel_166: three predict_house_price calls — NY 2500/3/2000, LA 3000/3/2005, Chicago 2000/2/1995. The second house is described as "similar" with only area/year/location changed, so rooms == 3 per ground truth.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 147 OUT1 073
✓	PredictProfitForTwoFundsBFCL/parallel_173: two investment_predictProfit calls — (5000,0.07,5) and (8000,0.05,7).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 916 OUT1 029
✓	RunChiSquaredTestForTwoContingencyTablesBFCL/parallel_164: two chi_squared_test calls — tables [45,55,35,65] and [30,70,50,50], alpha 0.05. Prompt states "significance level of 0.05 for both" -> alpha == 0.05 required (tightened). Table order is fixed.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 900 OUT1 301
✓	RunTTestAtTwoSignificanceLevelsBFCL/parallel_165: two t_test calls — same datasets, alpha 0.05 then 0.01 (both stated -> required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 146 OUT1 799
Tool Calling - Simple: Arts, Venues & Music (set 1) 8 passed	100.0%	IN50 785 OUT3 614
✓	SimpleConcertDetailsBeyonceBFCL/simple_python_286 (concert.get_details): artist=Beyonce, location=San Diego, date=April 2022.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 358 OUT306
✓	SimpleConcertInfoPopNyBFCL/simple_python_284 (concert_info.get): location=New York, date=next month, genre=Pop.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 333 OUT384
✓	SimpleGenerateMelodyC4BFCL/simple_python_291 (music_generator.generate_melody): key=C, start_note=C4, length=16, tempo=120 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 381 OUT256
✓	SimpleKeySignatureCsharpBFCL/simple_python_298 (music_theory.key_signature): key='C#', scale_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 318 OUT286
✓	SimpleMuseumHoursLouvreBFCL/simple_python_273 (museum.get_hours): name='Louvre Museum', location=Paris, day omitted or Monday.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 325 OUT412
✓	SimplePaintNeededWall30x12BFCL/simple_python_259 (calculate_paint_needed): coverage_rate=400, length=30, height=12.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL	Passed	IN6 381 OUT349
✓	SimpleReligionHistoryChristianity16cBFCL/simple_python_252 (get_religion_history): religion=Christianity, century=16, sort_by=importance, count=10.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 358 OUT1 237
✓	SimpleSculptureInfoPlensaBFCL/simple_python_263 (get_sculpture_info): artist_name='James Plensa', detail=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 331 OUT384
Tool Calling - Simple: Arts, Venues & Music (set 2) 8 passed	100.0%	IN50 547 OUT3 870
✓	SimpleComposeMelodyCFGBFCL/simple_python_292 (compose_melody): progression=[C,F,G], measures=4, instrument=Piano or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 319 OUT416
✓	SimpleConcertFindDetailsWeekndBFCL/simple_python_290 (concert.find_details): artist=The Weeknd, month=December, year=2022 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 297 OUT483
✓	SimpleFindConcertRockChicagoBFCL/simple_python_285 (find_concert): location=Chicago, price=100, genre=Rock.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 307 OUT337
✓	SimpleMuseumInfoMetBFCL/simple_python_274 (museum_info): museum_name=Metropolitan Museum of Art, info_type=opening_hours or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 295 OUT470
✓	SimpleMusicalScaleCsharpBFCL/simple_python_299 (musical_scale): key='C#'/'C sharp', scale_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 287 OUT603
✓	SimplePaintRequirementWallExclusionBFCL/simple_python_260 (paint_requirement.calculate): area {width=20,height=12}, paint_coverage=350, exclusion {type=window, area=15}.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN6 416 OUT464
✓	SimpleReligionHistoryChristianity300To400BFCL/simple_python_254 (get_religion_history): religion=Christianity, start_year=300, end_year=400, event_type=all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 322 OUT628
✓	SimpleSculptureDetailsDavidBFCL/simple_python_264 (sculpture.get_details): artist=Michelangelo, title=David, detail=size.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 304 OUT469
Tool Calling - Simple: Arts, Venues & Music (set 3) 9 passed	100.0%	IN60 197 OUT3 176
✓	SimpleChordProgressionKeyBFCL/simple_python_297 (music.theory.chordProgression): progression=[I,V,vi,IV], bools any.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 693 OUT296
✓	SimpleCompoundInterest10kBFCL/simple_python_269 (calculate_compound_interest): principle=10000, interest_rate=0.05, time=10, compounds_per_year=1 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 725 OUT427
✓	SimpleConcertSearchClassicalLaBFCL/simple_python_287 (concert.search): genre=classical, location=Los Angeles/LA, date='this weekend', price_range=cheap.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 692 OUT273
✓	SimpleCreateMixCMajorBFCL/simple_python_293 (music_composer.create_mix): scale='C Major', note_duration=quarter, track_length=180.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 715 OUT253
✓	SimpleGenerateCircleImageRedBFCL/simple_python_256 (generate_circle_image): radius=50, color=Red, background=white or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 684 OUT184
✓	SimpleMuseumWorkingHoursLouvreBFCL/simple_python_276 (museum_working_hours.get): museum=Louvre, location=Paris, day=Monday or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 670 OUT324
✓	SimpleRetrieveReligionInfoBuddhismBFCL/simple_python_253 (retrieve_religion_info): religion_name=Buddhism, detail_level=full.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 665 OUT749
✓	SimpleSculptureSearchChicago19cBFCL/simple_python_265 (sculpture_search): location=Chicago, time_frame='19th century', material=all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 682 OUT292
✓	SimpleTimeDifferenceSfSydneyBFCL/simple_python_250 (get_time_difference): place1=San Francisco/SF, place2=Sydney.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL	Passed	IN6 671 OUT378

Tool Calling - Irrelevance: Architecture & Artwork 9 passed

100.0%

IN31 133 OUT4 738

✓

NoToolForApartmentRentCostBFCL/irrelevance_158: apartment rental cost asked; only a new-construction cost estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 461 OUT386

✓

NoToolForBestJazzSingerBFCL/irrelevance_164: best Jazz singer asked; only a top-instrument-players search (singers are not instrument players) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 456 OUT474

✓

NoToolForCathedralCeilingDesignBFCL/irrelevance_157: how to design a cathedral ceiling asked; only a lookup of facts about an existing building is available (retrieves data, does not advise on design) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 457 OUT865

✓

NoToolForHolyBookDefinitionBFCL/irrelevance_144: definition of a holy book asked; only a search-within-a-named-holy-book tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 455 OUT352

✓

NoToolForLectureTeacherBFCL/irrelevance_167: teacher of upcoming lectures asked; only an upcoming-concert details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 458 OUT280

✓

NoToolForMostFamousSculptureBFCL/irrelevance_153: most famous sculpture asked; only a sculpting-tools lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 463 OUT523

✓

NoToolForPianoMaintenanceTipsBFCL/irrelevance_166: piano maintenance tips asked; only an instrument rental-prices lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 458 OUT573

✓

NoToolForScreamArtistBFCL/irrelevance_159: artist behind 'The Scream' asked. NOTE: the only tool (artwork_search) requires a museum_location that the user did not provide, so it cannot be correctly called; BFCL labels this irrelevant. Kept faithfully with ground truth = no tool call (flagged as borderline).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 462 OUT397

✓

NoToolForSculptureRockTypeBFCL/irrelevance_152: which rock suits a garden sculpture asked; only a sculpture 3D-model creator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 463 OUT888

Tool Calling - Irrelevance: Buildings, Sculpture & Music 7 passed1 skipped

100.0%

IN22 989 OUT3 136

–

NoToolForCelloInstrumentTypeBFCL/irrelevance_165: instrument classification of a cello asked. NOTE: BFCL labels this irrelevant, but the only tool (get_instrument_info with detail=type) arguably answers it; flagged for review as a borderline BFCL item. Kept faithfully with ground truth = no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_instrument_info' with detail='type' answers 'what type of instrument is a cello' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForDeclarationSignersBFCL/irrelevance_137: who signed the declaration asked; only a historical-event DATE lookup is available (returns a date, not the signatories) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 282 OUT519

✓

NoToolForEiffelTowerHeightBFCL/irrelevance_156: Eiffel Tower height asked; only an architecture-plan generator (creates a new design, does not retrieve facts) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 286 OUT307

✓

NoToolForFamousDancersBFCL/irrelevance_162: famous dancers asked; only a museum-visual-artists lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 288 OUT514

✓

NoToolForReformationInitiatorBFCL/irrelevance_145: who initiated the Reformation asked; only a religious-event YEAR lookup is available (returns a year, not a person) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 281 OUT469

✓

NoToolForScientistDefinitionBFCL/irrelevance_143: definition of "scientist" asked; only a named-historical-figure info lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 278 OUT406

✓

NoToolForSellingGuitarBFCL/irrelevance_163: how to sell a guitar asked; only an instrument-tuning tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 283 OUT498

✓

NoToolForSimilarBlueColorBFCL/irrelevance_148: a color similar to blue asked; only a complementary-color tool (opposite, not similar) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 291 OUT423

Tool Calling - Irrelevance: Cell Biology & Genetics 8 passed2 skipped

100.0%

IN30 315 OUT5 490

✓

NoToolForClottingBloodCellsBFCL/irrelevance_32: blood cell type responsible for clotting asked; only a nucleus-count-based cell-type classifier is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 790 OUT607

✓

NoToolForCommonAncestorBFCL/irrelevance_41: common ancestor of two animals asked (phylogenetics knowledge); only a fibonacci-sequence calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 789 OUT667

✓

NoToolForEvolutionaryAlgorithmInGameAIBFCL/irrelevance_43: how to apply an Evolutionary Algorithm in game AI asked (advice); only a creature-evolution simulator is available (wrong context).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 790 OUT1 173

✓

NoToolForGeneVariantFrequencyBFCL/irrelevance_36: gene-variant frequency in a population asked; only a dominant-trait-from-alleles tool is available (the true-match variant-frequency tool is deliberately isolated in A4).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 795 OUT666

–

NoToolForIdentifyingGeneticSequenceBFCL/irrelevance_33: interpret a genetic code sequence asked; only a species-identification-from-sequence tool is available (returns a species, not what the sequence means).0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'identify_species' (identifies a species from a genetic sequence) plausibly answers the query about a DNA sequence, so a competent model may call it - BFCL-mislabeled/borderline-relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForIncreasingDeerPopulationBFCL/irrelevance_39: how-to advice on growing an animal population asked; only a population-growth CALCULATOR (needs numeric rates) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 789 OUT718

✓

NoToolForLionMatingProcessBFCL/irrelevance_35: mating process of a species asked; only a dominant/recessive genetic-traits lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 786 OUT529

✓

NoToolForMitochondriaCountBFCL/irrelevance_29: number/count of mitochondria asked; only a cell-part FUNCTION lookup is available (wrong query direction).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 790 OUT526

✓

NoToolForPandaEvolutionaryHistoryBFCL/irrelevance_42: evolutionary history asked (a narrative fact); only a biodiversity-index calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 786 OUT604

–

NoToolForWhaleGeneSequenceBFCL/irrelevance_44: the actual gene sequence of a species asked; only a gene-sequence GENERATOR (needs numeric mutation rate + duration) is available.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'gene_sequencer' (generate gene sequences for evolutionary changes) directly answers the query - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Irrelevance: Color, Geometry & Population 11 passed

100.0%

IN40 836 OUT4 001

✓

NoToolForColorInComputerVisionBFCL/irrelevance_131: color meaning in computer vision asked; only a color-in-personality-psychology analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 712 OUT616

✓

NoToolForDeclarationSigningDateBFCL/irrelevance_135: signing date asked; only date-arithmetic (add days) and age tools are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 710 OUT276

✓

NoToolForElectricityDiscovererBFCL/irrelevance_140: who discovered electricity asked; only an electric-current calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 706 OUT310

✓

NoToolForHexColorMixBFCL/irrelevance_146: mixing two hex colors asked; only a prophet-details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 722 OUT561

✓

NoToolForIndependenceSigningDateBFCL/irrelevance_138: signing date asked; only an age calculator (needs birthdate + current date) and a date-adder are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 710 OUT300

✓

NoToolForMostVisitedMarketBFCL/irrelevance_161: most visited market asked; only a museum visitation-statistics lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 712 OUT334

✓

NoToolForNbaFinalWinnerBFCL/irrelevance_133: NBA final winner asked; only a historical-battle details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 713 OUT250

✓

NoToolForPicassoPaintingYearBFCL/irrelevance_151: year of a Picasso painting asked; only a sculpture-dimensions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 711 OUT316

✓

NoToolForRelativityAuthorBFCL/irrelevance_142: who proposed special relativity asked; only a chemical-element properties lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 715 OUT253

✓

NoToolForSculptureCreatorBFCL/irrelevance_155: sculpture creator asked; only a max-seating-capacity calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 711 OUT241

✓

NoToolForWhichColorsToMixBFCL/irrelevance_150: which colors to mix for a shade asked; only an RGB-to-Pantone converter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 714 OUT544

Tool Calling - Irrelevance: Cooking, Currency & Markets 8 passed

100.0%

IN25 943 OUT4 371

✓

NoToolForBakerySourdoughBFCL/irrelevance_223: a BAKERY selling sourdough asked; only a grocery-shop product finder available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 245 OUT805

✓

NoToolForChickenRecipeBFCL/irrelevance_211: the RECIPE to cook chicken asked; only a cook-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 241 OUT640

✓

NoToolForChocolateCakeIngredientsBFCL/irrelevance_220: cake INGREDIENTS asked; only a cooking-TEMPERATURE tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 244 OUT475

✓

NoToolForCurrencyToGbpBFCL/irrelevance_233: CURRENCY conversion asked; only a WEATHER tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 246 OUT253

✓

NoToolForHowToBoilAnEggBFCL/irrelevance_212: best WAY to boil an egg asked; only a boiling-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 243 OUT697

✓

NoToolForLasagnaRecipeBFCL/irrelevance_219: the RECIPE for lasagna asked; only a recipe-CALORIE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 241 OUT604

✓

NoToolForLegExercisesBFCL/irrelevance_221: leg EXERCISES asked; only a grocery-list generator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 241 OUT515

✓

NoToolForPetStoreBFCL/irrelevance_224: a PET store asked; only a GROCERY-store locator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 242 OUT382

Tool Calling - Irrelevance: Dates & Elements 10 passed

100.0%

IN36 270 OUT4 213

✓

NoToolForHydrogenPropertiesBFCL/irrelevance_141: chemical properties of Hydrogen asked; only a scientist-contributions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 626 OUT748

✓

NoToolForLargestPlanetBFCL/irrelevance_139: largest planet asked; only a star-info lookup (needs a star name) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 626 OUT364

✓

NoToolForMostImportantProphetBFCL/irrelevance_147: most important prophet asked; only a two-color mixer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 626 OUT296

✓

NoToolForMuseumVisitFrequencyBFCL/irrelevance_160: museum member visit frequency asked; only a single most-frequent-visitor lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 631 OUT314

✓

NoToolForPantoneOfSkyBlueBFCL/irrelevance_149: Pantone code of sky blue asked; only a paint-proportions calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 628 OUT449

✓

NoToolForPersonalityTypeBFCL/irrelevance_130: user asks what personality type they are; only a big-five calculator (which needs the user's rated answers) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 624 OUT371

✓

NoToolForStadiumSeatingCapacityBFCL/irrelevance_154: stadium seating capacity asked; only a sculpture-creator lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 627 OUT328

✓

NoToolForVicePresidentBFCL/irrelevance_136: Vice President asked; only a US-president-by-year lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 626 OUT375

✓

NoToolForWaterlooCasualtiesBFCL/irrelevance_132: casualty count of a battle asked; only a historical-event date lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 628 OUT650

✓

NoToolForWorldCupWinnerBFCL/irrelevance_134: sports World Cup winner asked; only a historical-battle outcome predictor is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 628 OUT318

Tool Calling - Irrelevance: Electromagnetism, Thermo & Data 9 passed1 failed

90.0%

IN37 554 OUT4 775

✓

NoToolForElectronRestMassBFCL/irrelevance_23: mass of an electron asked (a knowledge fact); only magnetic-field / current tools are available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 397 OUT288

✓

NoToolForFlightDurationBFCL/irrelevance_20: flight duration asked; only a currency converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 400 OUT313

✓

NoToolForFreezingPointBFCL/irrelevance_25: freezing point asked; only boiling-point calculators are available (wrong phase transition).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 407 OUT672

×

NoToolForGasAmountGeneratedBFCL/irrelevance_26: amount of gas generated asked; only a gas-PRESSURE calculator is available (wrong quantity).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 929 OUT1 231

✓

NoToolForGiniCoefficientBFCL/irrelevance_48: Gini coefficient asked; only a GDP-fetch tool is available (different economic indicator).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 400 OUT404

✓

NoToolForHeatingWaterBFCL/irrelevance_27: heat energy to warm water asked. NOTE: the original tool (calculate_heat) is a fair match for this query (only the well-known specific heat is missing), which would make "no call" ambiguous; per human-baseline fairness it is deliberately NOT shown here (kept as a distractor in A3), so no visible tool fits and "no call" is unambiguously correct.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 411 OUT512

✓

NoToolForHexagonSidesBFCL/irrelevance_28: number of sides of a hexagon asked (basic geometry knowledge); only a boiling-point tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 398 OUT45

✓

NoToolForSymbolicMagneticFieldBFCL/irrelevance_22: magnetic field asked with SYMBOLIC values (distance 'r', current 'I'); the two magnetic-field tools require concrete numeric distance + current, so there is nothing to pass -> no fit.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 411 OUT589

✓

NoToolForTimeDifferenceBFCL/irrelevance_49: time-zone difference asked; only a geo-coordinate lookup tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 400 OUT497

✓

NoToolForWorldSeriesWinnerBFCL/irrelevance_53: sports trivia asked; only a SQL database-query runner is available (no database/query provided).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 401 OUT224

Tool Calling - Irrelevance: Entertainment, Sports & Knowledge 9 passed

100.0%

IN31 410 OUT2 781

✓

NoToolForBasketballResultBFCL/irrelevance_84: basketball game result asked; only a stock-price retrieval tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 490 OUT220

✓

NoToolForBestSoccerPlayerBFCL/irrelevance_79: best current soccer player asked; only a stock-market prediction tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 488 OUT226

✓

NoToolForCaliforniaGovernorBFCL/irrelevance_87: current Governor of California asked; only a court-case filing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 486 OUT282

✓

NoToolForCrimeThrillerMovieListBFCL/irrelevance_88: best crime-thriller movies of 2020 asked; only a document forgery detector is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 495 OUT520

✓

NoToolForFifaWorldCupWinnerBFCL/irrelevance_80: FIFA World Cup 2010 winner asked; only a stock-ticker lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 491 OUT142

✓

NoToolForFrenchTranslationBFCL/irrelevance_61: French translation of a word asked; only an SVM-model tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 490 OUT140

✓

NoToolForHorrorMovieListBFCL/irrelevance_77: horror movie recommendations asked; only an investment future-value tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 489 OUT554

✓

NoToolForPresidentialElectionWinnerBFCL/irrelevance_85: 2020 presidential election winner asked; only a criminal-case detail lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 491 OUT237

✓

NoToolForSportsNewsBFCL/irrelevance_76: latest sports news asked; only a portfolio investment-trend analyzer is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 490 OUT460

Tool Calling - Irrelevance: Finance & Economics 8 passed

100.0%

IN26 826 OUT3 526

✓

NoToolForAverageHousePriceBFCL/irrelevance_69: market-wide average house price asked; only a regression predictor (needs a trained model + feature vector, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 346 OUT280

✓

NoToolForCompoundInterestBFCL/irrelevance_70: compound-interest computation asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 372 OUT834

✓

NoToolForGoldPriceBFCL/irrelevance_78: today's gold spot price asked; only a bond-price calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 348 OUT420

✓

NoToolForHorrorMovieCatalogBFCL/irrelevance_81: horror movie recommendations asked; only a historical stock-price fetcher is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 349 OUT585

✓

NoToolForLossProjectionBFCL/irrelevance_73: a loss projection asked; only a revenue-prediction tool is available (revenue != loss).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 351 OUT331

✓

NoToolForPoundToKilogramBFCL/irrelevance_75: a mass unit conversion asked; only a portfolio-returns calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 347 OUT126

✓

NoToolForProfitMarginBFCL/irrelevance_71: profit margin asked; only an ROI calculator is available (ROI != margin) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 369 OUT523

✓

NoToolForStockNewsBFCL/irrelevance_82: stock news asked; only a capital-gains calculator is available (gains != news) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 344 OUT427

Tool Calling - Irrelevance: Finance Near-Matches 9 passed

100.0%

IN30 921 OUT4 025

✓

NoToolForBitcoinPriceBFCL/irrelevance_56: current bitcoin price asked; only a generic database record-select tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 426 OUT302

✓

NoToolForBookstoreLookupBFCL/irrelevance_55: which online bookstore sells a title asked; only an add-to-cart tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 431 OUT302

✓

NoToolForBusinessRateOfReturnBFCL/irrelevance_74: rate of return asked; only a net-profit calculator is available (profit != rate of return).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 447 OUT666

✓

NoToolForCoinFlipProbabilityBFCL/irrelevance_66: probability of a coin landing heads asked; only a standard-deviation tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 431 OUT221

✓

NoToolForCurrentInterestRateBFCL/irrelevance_83: the current market interest rate asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 425 OUT306

✓

NoToolForExternalRateOfReturnBFCL/irrelevance_72: EXTERNAL rate of return asked; only an internal-rate-of-return (IRR) tool is available (ERR != IRR, even though cash flows are given) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 455 OUT518

✓

NoToolForFinalPriceWithoutBasePriceBFCL/irrelevance_60: final price after discount+tax asked, but the original price is never given, so the (domain-matching) calculateFinalPrice cannot be correctly called -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 445 OUT264

✓

NoToolForFrontendDevelopmentBFCL/irrelevance_62: how to build an e-commerce frontend asked; only a recommender-model builder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 431 OUT1 078

✓

NoToolForHighestGrossingMovieBFCL/irrelevance_54: highest-grossing movie of all time asked; only a movie search (requires a specific title + year, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 430 OUT368

Tool Calling - Irrelevance: Food, Time & Distance 4 passed1 skipped

100.0%

IN10 832 OUT1 175

✓

NoToolForCakeBakingMachineBFCL/irrelevance_218: a baking MACHINE/appliance asked; only a recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 706 OUT401

✓

NoToolForDrivingTravelTimeBFCL/irrelevance_215: driving TRAVEL TIME asked; only a restaurant-bill tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 710 OUT261

✓

NoToolForLondonTimezoneNameBFCL/irrelevance_226: the NAME of London's timezone asked; only a local-TIME-by-timezone tool available (it needs the timezone as input and returns a time, not the timezone name) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 702 OUT349

–

NoToolForPizzaInBostonBFCL/irrelevance_213: a place for pizza asked; a cuisine+location restaurant_finder is available. REVIEW FLAG (fairness): restaurant_finder(cuisine=pizza/Italian, location=Boston) plausibly answers this, so a competent human might legitimately call it. BFCL labels it irrelevant; kept as such and marked Hard because the intended "no call" is debatable. No OTHER query here is a restaurant search, so the tool cannot wrongly satisfy the file's other cases.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'restaurant_finder' (cuisine+location) answers 'good place for pizza in Boston' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForTriangleHypotenuseBFCL/irrelevance_238: the HYPOTENUSE of a triangle asked; only a geo lat/long distance calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 714 OUT164

Tool Calling - Irrelevance: Geo, Travel & Sentiment 7 passed1 failed1 skipped

87.5%

IN30 622 OUT3 352

✓

NoToolForAverageTemperatureBmiBFCL/irrelevance_110: an average December temperature asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 396 OUT296

✓

NoToolForCaseLawyerBFCL/irrelevance_95: the lawyer of a case asked; only a case-outcome retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 399 OUT428

✓

NoToolForCurrentTimeSunriseBFCL/irrelevance_102: the current time asked; only a sunrise/sunset time calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 395 OUT274

✓

NoToolForFamousQuoteBFCL/irrelevance_119: a famous quote asked; only a sentiment-analysis tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 393 OUT93

✓

NoToolForFastestRouteBFCL/irrelevance_114: the fastest route asked; only a closest-airport finder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 397 OUT697

–

NoToolForRestaurantsInLondonBFCL/irrelevance_113: restaurants in London asked; only a generic points-of-interest locator is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'find_pois' (locate points of interest by category) answers 'restaurants in London' (restaurants are POIs) - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForSportsBrainImpactBFCL/irrelevance_120: the neurological impact of sports asked; only a caffeine neurological-impact tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 396 OUT455

✓

NoToolForTrafficConditionBFCL/irrelevance_112: the current traffic condition asked; only an address geocoder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 395 OUT371

×

NoToolForTravelTimeByCarBFCL/irrelevance_115: travel time by car asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 851 OUT738

Tool Calling - Irrelevance: Legal & Case Lookup 10 passed

100.0%

IN35 507 OUT3 594

✓

NoToolForAppleCaloriesBFCL/irrelevance_100: calories in an apple asked; only a litigation-cost calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 548 OUT261

✓

NoToolForBrainWeightBFCL/irrelevance_122: average human brain weight asked; only a per-patient neural-activity tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 550 OUT236

✓

NoToolForCaseJudgementBFCL/irrelevance_93: a case judgement asked; only a law-firm impactful-cases lister is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 548 OUT328

✓

NoToolForChineseRestaurantBFCL/irrelevance_97: a Chinese-restaurant recommendation asked; only a lawsuit-filing tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 551 OUT396

✓

NoToolForMovieRecommendationBFCL/irrelevance_116: a movie recommendation asked; only a word-count tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 549 OUT358

✓

NoToolForNbaChampionBFCL/irrelevance_118: the 1996 NBA champion asked; only an NBA playoff-brackets display tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 551 OUT506

✓

NoToolForPaintingEiffelSettlementBFCL/irrelevance_98: how long to paint the Eiffel Tower asked; only a lawsuit-settlement estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 552 OUT244

✓

NoToolForSecurityGuardSearchBFCL/irrelevance_92: finding a security guard asked; only a lawyer-search tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 558 OUT456

✓

NoToolForSphereVolumeBFCL/irrelevance_104: sphere volume asked; only a circular-park area tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 552 OUT378

✓

NoToolForTrafficLawsBFCL/irrelevance_99: traffic laws asked; only a lawsuit-search tool is available (lawsuits, not laws) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 548 OUT431

Tool Calling - Irrelevance: Math & Science Distractors 8 passed

100.0%

IN25 236 OUT2 116

✓

NoToolForDefiniteIntegralBFCL/irrelevance_9: definite integral asked; only an income-tax calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 157 OUT324

✓

NoToolForIntegralOfCubeBFCL/irrelevance_8: definite integral asked; only a string-to-int converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 150 OUT319

✓

NoToolForLatestMoviesBFCL/irrelevance_15: latest movie releases asked; only a physics final-velocity tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 144 OUT311

✓

NoToolForPerpendicularSlopeBFCL/irrelevance_3: slope of a perpendicular line asked; only a critical-points finder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 162 OUT217

✓

NoToolForPrimeFactorsBFCL/irrelevance_13: prime factorization asked; only a compound-interest calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 147 OUT255

✓

NoToolForRectangleAreaBFCL/irrelevance_6: rectangle area asked; only a circle-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 158 OUT106

✓

NoToolForTriangleAreaBFCL/irrelevance_0: triangle area asked; only a BMI tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 158 OUT247

✓

NoToolForUpwardAccelerationBFCL/irrelevance_14: acceleration asked (it is just gravity); the tempting max-height and final-velocity tools compute neither the acceleration nor what is asked -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 160 OUT337

Tool Calling - Irrelevance: Math & Statistics 5 passed1 failed

83.3%

IN21 146 OUT3 224

✓

NoToolForBinomialMeanBFCL/irrelevance_67: the MEAN of a binomial experiment asked; only p-value tools are available (they return a p-value, not the mean) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 026 OUT783

✓

NoToolForCivilLawComponentsBFCL/irrelevance_90: the components/categories of Civil Law asked; only a noise-complaint filing tool is available (filing a complaint does not explain law) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN2 998 OUT575

✓

NoToolForCoinTossMaxHeadsBFCL/irrelevance_63: how many heads CAN be obtained from tossing 3 coins (a count/maximum, not a probability) asked; only a probability-of-an-event calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 002 OUT176

×

NoToolForNullHypothesisStdDevBFCL/irrelevance_68: the STANDARD DEVIATION of a null-hypothesis test asked; only p-value tools are available (they return a p-value, not a standard deviation) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 112 OUT892

✓

NoToolForPowerOfTwoBFCL/irrelevance_59: raising a number to a power asked; only a linear-equation solver is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 001 OUT155

✓

NoToolForQuadraticEquationBFCL/irrelevance_57: solving a quadratic equation asked; only a genetic-algorithm optimizer and a LINEAR equation solver are available (a quadratic is not linear) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 007 OUT643

Tool Calling - Irrelevance: Medical, Social & Personality 6 passed3 skipped

100.0%

IN20 452 OUT2 663

✓

NoToolForBigMacCaloriesBFCL/irrelevance_123: calories of a Big Mac asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 407 OUT275

✓

NoToolForEconomicHappinessEffectBFCL/irrelevance_126: the effect of economic status on happiness asked; only a happiness-index lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 409 OUT569

✓

NoToolForFriendMbtiBFCL/irrelevance_129: a friend's MBTI asked; only a personality-type evaluator (needs trait percentages) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 407 OUT369

–

NoToolForLatestTechTrendBFCL/irrelevance_124: the latest technology trend asked; only a trending-topics-by-category tool is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_social_trends' (trending topics by category) answers 'latest trend in technology' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForMajorAirportsBFCL/irrelevance_117: major US airports asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 410 OUT490

–

NoToolForMotorNeuronDiseaseBFCL/irrelevance_121: motor neuron disease info asked; only a generic disease-info retriever is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'medical_records_get_disease_info' directly answers 'information on motor neuron diseases' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForPopularBooksBFCL/irrelevance_125: popular books by an author asked; only a recent-tweets retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 409 OUT574

–

NoToolForTwitterMoodBFCL/irrelevance_127: overall twitter mood asked; only a twitter sentiment-analysis tool is available -> BFCL ground truth is no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'sentiment_analysis_twitter' directly answers 'general mood of twitter regarding the new iPhone' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForVegetableServingsBFCL/irrelevance_128: daily vegetable servings asked; only a personality-test scorer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 410 OUT386

Tool Calling - Irrelevance: Mixed Distractors 8 passed

100.0%

IN25 992 OUT2 905

✓

NoToolForAirQualityBFCL/irrelevance_40: current air quality asked; only a plant-biomass calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 250 OUT302

✓

NoToolForFavoriteRestaurantBFCL/irrelevance_31: a favorite restaurant recommendation asked; only a cell-division simulator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 246 OUT264

✓

NoToolForGlobalWarmingDocumentaryBFCL/irrelevance_38: a documentary (video) asked; only a scientific-research-paper retriever is available (wrong medium).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 247 OUT411

✓

NoToolForLasagnaIngredientsBFCL/irrelevance_47: recipe ingredients asked; only a flight-timetable tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 247 OUT508

✓

NoToolForLionDominantTraitBFCL/irrelevance_34: dominant genetic trait of a species asked; only a gene-variant-FREQUENCY lookup is available (different genetic output).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 249 OUT579

✓

NoToolForMultinucleatedCellNameBFCL/irrelevance_30: name of a cell type with multiple nuclei asked; only a blood-cell classifier (by shape/size) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 253 OUT362

✓

NoToolForPicnicSpotBFCL/irrelevance_37: a picnic spot asked; only a local-fauna information tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 246 OUT378

✓

NoToolForTranslationBFCL/irrelevance_52: natural-language translation asked; only a file-compression tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 254 OUT101

Tool Calling - Irrelevance: Motion, Electricity & Misc 10 passed

100.0%

IN35 408 OUT5 124

✓

NoToolForCarProjectileRangeBFCL/irrelevance_16: projectile-style "how far will a car travel" asked with symbolic v/t/theta; the range tool needs concrete numeric velocity + angle -> nothing to pass -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 551 OUT693

✓

NoToolForConeVolumeBFCL/irrelevance_19: cone volume asked; only an investment-return calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 545 OUT412

✓

NoToolForCurrentClockTimeBFCL/irrelevance_17: current clock time asked; only a travel-time (distance/speed) calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 534 OUT241

✓

NoToolForElectronMassBFCL/irrelevance_24: mass of an electron asked (a knowledge fact); only an electric-current tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 535 OUT290

✓

NoToolForFibonacciOf20BFCL/irrelevance_51: fibonacci number asked; only a cryptocurrency-price tool is available (the fibonacci calculator lives in a different file so it cannot satisfy this query).1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 536 OUT656

✓

NoToolForForceAngleFromMomentumBFCL/irrelevance_18: angle of a force from momentum asked; only a vector-angle-from-XY-components tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 541 OUT1 120

✓

NoToolForHorrorMovieListBFCL/irrelevance_50: movie recommendations asked; only a Celsius-to-Fahrenheit converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 541 OUT563

✓

NoToolForMagneticFieldNearWireBFCL/irrelevance_21: magnetic field near a current-carrying wire asked; only an EM-wave-amplitude tool is available (magnetic-field calculators are deliberately kept in a different file).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 550 OUT541

✓

NoToolForSineOf45DegreesBFCL/irrelevance_45: sine of an angle asked; only a polygon-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 536 OUT286

✓

NoToolForTeslaPriceBFCL/irrelevance_46: product price of a car asked; only a currency-exchange-rate tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 539 OUT322

Tool Calling - Irrelevance: Music Theory & Lookup 8 passed

100.0%

IN25 572 OUT3 889

✓

NoToolForAncientGreekLawmakersBFCL/irrelevance_171: who established laws in Ancient Greece asked (history); only a musical-piece composer search is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 194 OUT721

✓

NoToolForBasketballGameAtVenueBFCL/irrelevance_169: who is playing a basketball game at a venue asked; only concert-locating tools exist.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 197 OUT321

✓

NoToolForBluesScalePatternBFCL/irrelevance_176: the interval pattern of a blues scale asked; only composer lookups are available (get_blues_scale lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 199 OUT715

✓

NoToolForDonQuixoteAuthorBFCL/irrelevance_172: author of the novel Don Quixote asked; only music-composition lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 192 OUT349

✓

NoToolForFamousUsComposersBFCL/irrelevance_170: most famous US composers asked (general knowledge, no specific piece); only a chord-progression creator + piece-specific composer lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 195 OUT502

✓

NoToolForGrammyBestAlbumBFCL/irrelevance_177: Grammy Award winner asked; only a song chord-progression lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 200 OUT528

✓

NoToolForPrimaryTriadsOfKeyBFCL/irrelevance_173: primary triads of a key asked; only a genre-common-chords finder is available (primary_triads lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 198 OUT441

✓

NoToolForUniversityClassAvailabilityBFCL/irrelevance_168: availability of a university class asked; only concert-availability / concert-search tools are available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 197 OUT312

Tool Calling - Irrelevance: Probability & Legal 5 passed

100.0%

IN13 997 OUT3 025

✓

NoToolForFaceCardProbabilityBFCL/irrelevance_64: probability of drawing a face card from a deck asked; only a COIN-toss heads probability tool is available (coins, not cards) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 796 OUT351

✓

NoToolForHomeElectricityNeedBFCL/irrelevance_58: home electricity demand (kWh) asked; only a solar-panel COUNT calculator is available (it sizes panels, not electricity need, and needs sunlight hours not given) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN2 799 OUT719

✓

NoToolForNoiseComplaintProcessBFCL/irrelevance_91: how/whether to report a noise complaint asked; only a law-category listing tool is available (it lists legal categories, it does not file or advise on complaints) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 796 OUT481

✓

NoToolForRedMarbleCountBFCL/irrelevance_65: number of red marbles asked; only a determine-population tool is available (it returns a total population from probability + sample size, not a sub-category count) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN2 811 OUT530

✓

NoToolForTenantRightsBFCL/irrelevance_89: explaining tenant rights asked; only a contract-generation tool is available (generating a contract does not explain rights) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN2 795 OUT944

Tool Calling - Irrelevance: Recipes & Cooking 8 passed1 skipped

100.0%

IN27 642 OUT2 870

✓

NoToolForBestGameCharacterBFCL/irrelevance_206: best game character vs a dragon asked; only a best-WEAPON finder available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 459 OUT445

✓

NoToolForBestMovie2020BFCL/irrelevance_235: best MOVIE of 2020 asked; only a currency-exchange calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 456 OUT299

✓

NoToolForBuildingAPcBFCL/irrelevance_209: how to build a PC asked; only a cooking-recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 451 OUT631

✓

NoToolForFamousPlaceInParisBFCL/irrelevance_210: most famous PLACE in Paris asked; only a restaurant-by-dish search available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 453 OUT400

✓

NoToolForGameShortestPathBFCL/irrelevance_208: shortest in-game path asked; only a taxi-fare calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 461 OUT389

✓

NoToolForInchToMeterBFCL/irrelevance_234: LENGTH conversion (inches to metres) asked; only a CURRENCY converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 452 OUT232

✓

NoToolForLowestGameScoreBFCL/irrelevance_207: LOWEST game score asked; only a HIGHEST-score retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 456 OUT211

✓

NoToolForNewYorkPopulationBFCL/irrelevance_225: current POPULATION of New York asked; only a time-zone converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 454 OUT263

–

NoToolForQuickestFlightLondonTokyoBFCL/irrelevance_236: quickest way to fly London->Tokyo asked; a flight-duration tool is available. REVIEW FLAG (fairness): get_flight_duration plausibly answers this ("quickest ... by plane" ~= quickest flight duration), so a competent human might legitimately call it. Kept as BFCL labels it (irrelevant), but this item is borderline. Marked Hard because the intended "no call" is debatable.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_flight_duration' (with flight_type non-stop) plausibly answers 'quickest way ... by plane' - BFCL-mislabeled/borderline-relevant. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Irrelevance: Restaurants & Travel 8 passed

100.0%

IN26 515 OUT2 531

✓

NoToolForBeijingSunsetBFCL/irrelevance_227: SUNSET time asked; only a SUNRISE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 313 OUT139

✓

NoToolForClosestItalianRestaurantBFCL/irrelevance_216: FIND the closest Italian restaurant asked; only a currency converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 312 OUT175

✓

NoToolForFindSushiRestaurantBFCL/irrelevance_214: FIND a sushi restaurant asked; only a tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 314 OUT242

✓

NoToolForNearestPharmacyBFCL/irrelevance_237: FIND the nearest pharmacy asked; only a route-to-a-known-location planner available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 314 OUT187

✓

NoToolForPetFriendlyTrainStationBFCL/irrelevance_230: a pet-friendly TRAIN STATION asked; only hotel search tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 313 OUT297

✓

NoToolForSushiRestaurantsTokyoBFCL/irrelevance_229: FIND sushi RESTAURANTS asked; only hotel-booking tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 314 OUT532

✓

NoToolForThaiRestaurantChicagoBFCL/irrelevance_231: FIND a Thai RESTAURANT asked; only a hotel-availability checker available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 315 OUT357

✓

NoToolForYogaRetreatBFCL/irrelevance_232: a yoga RETREAT asked; only a HOTEL search tool available. REVIEW FLAG (fairness): a retreat is not a hotel, but hotel_search_find_hotels accepts arbitrary amenities (spa, vegetarian) and a location, so it is somewhat tempting. BFCL labels this irrelevant (retreat != hotel); kept as such and marked Hard because the boundary is borderline.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 320 OUT602

Tool Calling - Irrelevance: Sports & Games (set 1) 10 passed

100.0%

IN35 688 OUT4 455

✓

NoToolForAssistAverageBFCL/irrelevance_185: assist average asked; only an average-SCORING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 570 OUT221

✓

NoToolForBestChessMoveBFCL/irrelevance_194: best next move asked; only a board-game-STATISTICS tool is available (the move-suggesting board_analyser lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 572 OUT451

✓

NoToolForChessArrangementCountBFCL/irrelevance_196: total theoretical arrangements asked; only a final-score calculator is available (possible_moves lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 572 OUT636

✓

NoToolForFootballTeamRankingBFCL/irrelevance_186: ranking of a team asked; only a season-MVP tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 568 OUT451

✓

NoToolForIceCreamStoreBFCL/irrelevance_201: find a physical store asked; only a video-game guide is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 565 OUT374

✓

NoToolForMostHeartsInPokerBFCL/irrelevance_199: who holds most of a suit asked; only a poker-dealing tool is available (get_highest_card_holder lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 572 OUT363

✓

NoToolForPlayerBiographyBFCL/irrelevance_189: a biographical "who is X" asked; only a top-ranked-athlete lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 565 OUT480

✓

NoToolForTeamRosterBFCL/irrelevance_192: full roster asked; only a top-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 567 OUT352

✓

NoToolForUnoRulesBFCL/irrelevance_198: the rules of a game asked; only a deck-shuffling tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 569 OUT665

✓

NoToolForWorldCupWinnerBFCL/irrelevance_191: who won a tournament asked (no team named); only a per-team match-stats tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 568 OUT462

Tool Calling - Irrelevance: Sports & Games (set 2) 8 passed1 failed

88.9%

IN34 674 OUT3 795

✓

NoToolForBlackjackAceRuleBFCL/irrelevance_200: a rules question asked; only a poker highest-card-holder tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 467 OUT649

✓

NoToolForBookAuthorBFCL/irrelevance_205: author of a book asked; only a video-game score calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 471 OUT67

✓

NoToolForChessTournamentWinnerBFCL/irrelevance_195: who won a tournament asked; only a board-position move-suggester is available (the game-history fetch_game_stats lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 468 OUT389

✓

NoToolForCreateJigsawBFCL/irrelevance_204: CREATE a puzzle asked; only a puzzle-SOLVING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 461 OUT377

×

NoToolForHaloPlayerRankBFCL/irrelevance_203: a player's RANK asked; only a player-SCORE lookup is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 947 OUT398

✓

NoToolForLakersTopScorerBFCL/irrelevance_193: top scorer of a team asked; only a team-details tool (roster/results/upcoming, not scorers) is available (sports_team.get_top_scorer lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 466 OUT634

✓

NoToolForMonopolyWinnerBFCL/irrelevance_197: who won a casual game asked; only a possible-moves calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 466 OUT386

✓

NoToolForTopTennisPlayerBFCL/irrelevance_190: top ranked PLAYER asked; only a TEAM standing/ranking tool is available (the athlete-ranking get_top_ranked lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 465 OUT305

✓

NoToolForWorldSeriesWinnerBFCL/irrelevance_202: who won a real game asked; only a score calculator (from raw points) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 463 OUT590

Tool Calling - Irrelevance: Sports Statistics 9 passed

100.0%

IN31 560 OUT4 351

✓

NoToolForBasketballSeasonMvpBFCL/irrelevance_187: who won the MVP asked; only a team-ranking tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 509 OUT273

✓

NoToolForClippersRosterBFCL/irrelevance_179: roster of a team asked; only a two-team game-results tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 502 OUT821

✓

NoToolForCommonChordsInPopBFCL/irrelevance_174: most common chords in a pop song asked; only a key-signature primary-triads tool is available (find_common_chords lives in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 507 OUT823

✓

NoToolForCricketMatchParticipantsBFCL/irrelevance_180: who is playing in cricket matches today asked; only a cricket-schedule tool (dates/matches, not participants) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 506 OUT452

✓

NoToolForLaLigaPlayersBFCL/irrelevance_181: who played in a whole league asked; only a two-named-team last-match-result tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 502 OUT418

✓

NoToolForMoonlightSonataComposerBFCL/irrelevance_175: composer of Moonlight Sonata asked; only a blues-scale generator is available here (all composer-lookup tools are isolated in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 505 OUT327

✓

NoToolForNbaMvpWinnerBFCL/irrelevance_184: who won an MVP award asked (no player named); only a per-player stats tool that requires a player name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 511 OUT614

✓

NoToolForTopAssistPlayerBFCL/irrelevance_178: top ASSIST player asked; only a top-GOAL-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 506 OUT231

✓

NoToolForWimbledonChampionBFCL/irrelevance_183: winner of a tournament asked (no person named); only a celebrity-info tool that requires a name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 512 OUT392

Tool Calling - Irrelevance: Time, Weather & Misc 2 passed1 failed1 skipped

66.7%

IN10 342 OUT1 745

×

NoToolForCityDistanceKmBFCL/irrelevance_239: DISTANCE (km) between two cities asked; only a get_date tool that returns the TIME difference between two locations is available (its 'unit' arg mentions distance, but it computes time) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN5 188 OUT900

–

NoToolForCurrentTimeSydneyBFCL/irrelevance_228: current time in Sydney asked; a local-time-by-city tool is available. REVIEW FLAG (fairness): get_local_time(location='Sydney') directly answers this, so a competent human would legitimately call it. BFCL nonetheless files it under irrelevance; kept as such to preserve the source label, but this item is effectively RELEVANT and should be reviewed. Marked Hard because the "no call" expectation is contradicted by the tool itself.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_local_time' directly answers 'current time in Sydney' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForTomatoCaloriesBFCL/irrelevance_222: CALORIES in a tomato asked; only a grocery-item-details lookup available. REVIEW FLAG (fairness): item_details at 'detailed' level could arguably surface nutrition info, and it requires a store_location the user never gave; BFCL labels it irrelevant (it is a grocery-store item lookup, not a nutrition-facts service). Kept as such; marked Medium as borderline.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 578 OUT641

✓

NoToolForWriteABookBFCL/irrelevance_217: "can you write a book" asked; only a recipe-CREATION tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN2 576 OUT204

Tool Calling - Irrelevance: Weather, Environment & Services 10 passed

100.0%

IN36 617 OUT5 315

✓

NoToolForBestCo2ReductionBFCL/irrelevance_108: the best way to reduce CO2 asked; only a numeric emission-reduction estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 660 OUT465

✓

NoToolForBestMonthToVisitBFCL/irrelevance_101: best month to visit Hawaii asked; only an average-monthly-temperature tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 658 OUT683

✓

NoToolForCactusNutrientsBFCL/irrelevance_109: a plant's nutrient needs asked; only a weekly watering-needs calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 665 OUT380

✓

NoToolForCurrentTimeWeatherBFCL/irrelevance_103: the current time asked; only a current-weather forecast tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 659 OUT319

✓

NoToolForGardenSpeciesCompositionBFCL/irrelevance_107: garden species composition asked; only a soil sand/silt/clay composition analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 662 OUT545

✓

NoToolForGmoYogaMatBFCL/irrelevance_111: a GMO yoga mat asked; only an eco-friendly product locator is available (GMO is not eco-friendly) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 661 OUT578

✓

NoToolForImpactfulCasesBFCL/irrelevance_94: a firm's impactful cases asked; only a single-case detail lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 669 OUT533

✓

NoToolForPaintingEiffelCaseResultBFCL/irrelevance_96: how long to paint the Eiffel Tower asked; only a law-case result lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 661 OUT423

✓

NoToolForPollinationFlowersBFCL/irrelevance_105: top pollination flower species asked; only a route elevation-profile plotter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 663 OUT861

✓

NoToolForTomatoFertilizerBFCL/irrelevance_106: best fertilizer for tomatoes asked; only a soil-type analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 659 OUT528

Tool Calling - Parallel Calls: Art, Music & Local Services 10 passed1 skipped

100.0%

IN75 758 OUT7 196

–

CalculateDrivingDistancesShortestRouteBFCL/parallel_38: two calc_distance calls — New York→Washington DC and Los Angeles→San Francisco. Tightened per methodology: the prompt explicitly enables "shortest route", so shortest_route == true is required (schema default is false).0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Skipped: Ambiguous scope (item flaw, not model behavior): the trailing 'with optional parameter shortest route enabled' can reasonably apply to only the second leg (its nearest clause) or to both; the BFCL ground truth requires it on both, but a competent human would not answer deterministically. Kept for traceability.

Skipped

IN0 OUT0

✓

CheckInstrumentAvailabilityInTwoCitiesBFCL/parallel_42: two check_instrument_availability calls — Yamaha P125 in Berlin and Madrid.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 484 OUT387

✓

FindMusicStoresInTwoCitiesBFCL/parallel_41: two find_music_instrument_store calls — San Francisco and Chicago; instruments {Yamaha acoustic guitar, Kawai piano} in any order and either casing.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 527 OUT531

✓

FindRockAndJazzConcertsInTwoCitiesBFCL/parallel_43: four concert_finder calls — {San Francisco, New York} x {rock, jazz} over the next month. time_period may be omitted (defaults to 30 days ≈ "next month"). Isolated from concert_find_nearby (parallel_44) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 826 OUT746

✓

GetExhibitionDetailsForTwoMuseumsBFCL/parallel_40: two museum_exhibition_detail calls — 'Wonder of Nature' at the Louvre and 'Age of Reptiles' at the British Museum. visitor_type {child, adult} (unordered). Tightened per methodology: the prompt explicitly asks for cost "for children and adult", so both visitor types are required (schema default is ['adult'] only).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 548 OUT1 218

✓

GetMostPlayedPopAndRockSongsBFCL/parallel_45: two musicCharts_getMostPlayed calls — most played Pop and Rock song in Australia. Optional duration omitted (ground truth accepts the schema default of 0).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 470 OUT630

✓

GetMuseumInfoForTwoCitiesBFCL/parallel_39: two museum_info_get_info calls — Washington D.C. and Paris; details {Opening hours, Adult tickets, Child tickets} in any order.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 500 OUT462

✓

GetRgbAndHexForThreeColorsBFCL/parallel_37: three color_converter_get_color_info calls — navy, purple, maroon; each with {RGB, HEX} (unordered).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 646 OUT617

✓

GetTeamRankingsInTwoLeaguesBFCL/parallel_47: two get_team_ranking calls — Barcelona in UEFA Champions League and Manchester United in La Liga.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 471 OUT451

✓

MixPaintColorsForTwoRequestsBFCL/parallel_36: two paint_color_mixture calls — watercolor magenta and acrylic navy.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 520 OUT1 477

✓

WinningPercentageForTwoTeamsTwoSeasonsBFCL/parallel_46: four calculate_winning_percentage calls — {Lakers, Bulls} x {2018, 2020}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 766 OUT677

Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 1) 13 passed

100.0%

IN111 888 OUT18 502

✓

AverageFourNumberSetsBFCL/parallel_132: four calculate_average calls — one per list of numbers (element order preserved).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 785 OUT730

✓

CalculateBmiFourPeopleMetricBFCL/parallel_134: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt states "All measurements are in the metric system", so unit == "metric" is required (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 803 OUT723

✓

CalculateCityDistancesInKilometersBFCL/parallel_135: three geo_distance_calculate calls — NY->LA->Miami->NY, in kilometers. Tightened: the tool defaults to miles but the prompt asks "in kilometers", so units == "kilometers" is REQUIRED (omission would return miles, contradicting the request — the ground truth's "omit" option is unsafe here).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 523 OUT2 270

✓

CalculateGpsDistancesAlongRouteBFCL/parallel_133: three calculate_distance calls over GPS coordinate pairs (Paris->Rome->Athens-> Giza), each in kilometers. Uses the coordinate-based tool (coord1/coord2), not the city-name distance tools that are also visible.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 827 OUT6 764

✓

CompareAverageTemperatureNyAndLaBFCL/parallel_128: two average_temperature calls — New York 7 days in Fahrenheit, Los Angeles 7 days in Celsius. Tightened: the prompt explicitly requests Fahrenheit for NY, so temp_unit == "Fahrenheit" is required (also the default); Celsius is non-default and required for LA.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 389 OUT555

✓

CreateTwoHistogramsWithFiveBinsBFCL/parallel_129: two create_histogram calls — one per data set, 5 bins each (data order preserved).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 822 OUT710

✓

EstimateElephantPopulationThreeScenariosBFCL/parallel_125: three elephant_population_estimate calls — current 500 at 2%, 1.5%, 2.5% over 10y.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 699 OUT2 429

✓

FindItalianRestaurantsInNyAndLaBFCL/parallel_130: two find_restaurants calls — New York and Los Angeles, Italian, 4 results each, dietary_requirements [vegan, gluten-free]. Distinct from restaurant_find_nearby (this tool carries food_type + number, which the prompt supplies, so it is the correct choice here).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 422 OUT651

✓

FindRestaurantsByDietInThreeCitiesBFCL/parallel_127: three restaurant_find_nearby calls — New York, Los Angeles, Chicago, each with dietary_preference [Vegan, Gluten-free, Dairy-free]. List order follows the prompt (matches ground truth).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 545 OUT518

✓

PredictEvolutionDarwinAndLamarckBFCL/parallel_126: two prediction_evolution calls — African Elephant over 5000y, Darwin then Lamarck. Tightened per methodology: the prompt explicitly says "using the Darwin model", so model == "Darwin" is required for the first call (ground truth also accepts omission via the 'Darwin' default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 433 OUT534

✓

ProjectPopulationGrowthJapanAndIndiaBFCL/parallel_124: four population_projections calls — Japan/10y and India/20y, each at the current (default) growth rate and at an explicit rate. "current growth rate" maps to the default (growth_rate omitted, accepts null or 0.01); the explicit 1.5%/2.1% rates are required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 706 OUT1 463

✓

RouteThreeLegsAvoidingTollsBFCL/parallel_131: three map_routing_fastest_route calls — SF->Palo Alto->San Jose->home, all avoiding tolls. Tightened: the prompt explicitly says "avoiding toll roads", so avoid_tolls == true is required for every leg (non-default value stated in the prompt).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 551 OUT677

✓

ShortestCityDistanceBusWithAndWithoutTransferBFCL/parallel_136: two city_distance_find_shortest calls — NY->LA by bus, first without transfer then allowing transfer. Tightened: the prompt explicitly says "using a bus", so transportation == "bus" is required for both (also the default). allow_transfer: false/omitted for the first, true for the second.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 383 OUT478

Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 2) 14 passed

100.0%

IN125 790 OUT11 282

✓

CalculateAreaOfThreeTrianglesBFCL/parallel_143: three calc_area_triangle calls — (10,5), (15,7), (20,10).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 958 OUT585

✓

CalculateBinomialProbabilityBasketballBFCL/parallel_161: three calculate_binomial_probability calls — basketball shooter at 60%: (10,7), (15,10), (20,15). probability_of_success == 0.6 is required (prompt states 60%). ISOLATED here (away from prob_dist.binomial in ToolCalling-Parallel-D3): both tools compute binomial probability, and this prompt does not name a function, so having both visible would make the choice genuinely ambiguous. Kept apart so exactly one correct tool is available.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 037 OUT2 423

✓

CalculateBmiInMetersForThreePeopleBFCL/parallel_138: three calculate_BMI calls (weight kg, height meters) — John, Sarah, Mike.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 019 OUT706

✓

CalculateCelestialDistancesInMilesBFCL/parallel_149: two calculate_distance calls between celestial bodies (Mars-Venus, Mars-Jupiter), in miles. Uses the body-name distance tool (body1/body2). Tightened: prompt asks "in miles", so unit == "miles" is required (tool default is kilometers, so omission would be wrong).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 764 OUT522

✓

CalculateClockAnglesAtThreeTimesBFCL/parallel_145: three calculate_clock_angle calls — 3:15, 8:20, 11:50. Tightened: the prompt asks for results "rounded to 2 decimal places", so round_to == 2 is required (also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 939 OUT894

✓

CalculateFactorialOfFourNumbersBFCL/parallel_144: four math_factorial calls — 5, 3, 4, 2.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 968 OUT388

✓

CalculateLightTravelTimeThreeLegsBFCL/parallel_147: three light_travel_time calls — 4.22, 6.1, 5.88 light years. Tightened: the prompt explicitly states the speed of light (299792458 m/s), so speed_of_light == 299792458 is required (also default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 025 OUT1 614

✓

CalculateSpeedForCarBicycleTrainBFCL/parallel_148: three calculate_speed calls — car 500m/25s in km/h, bicycle 1000m/200s in m/s, train 10000m/600s in km/h. Each output unit is explicitly named in the prompt, so to_unit is required.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 040 OUT646

✓

FetchAllDataFieldsForTwoEmployeesBFCL/parallel_139: two employee_fetch_data calls — Tech Solutions, employees 12345 and 67890, each fetching all four data fields (order follows the prompt / ground truth).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 872 OUT447

✓

FindDiCaprioDramaAndComedyMoviesBFCL/parallel_140: two imdb_find_movies_by_actor calls — Leonardo DiCaprio, Drama in 2010 and Comedy in 2012. Tightened: the prompt explicitly names each category, so category is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 760 OUT749

✓

GetImaxAndTwoDTheaterReleasesBFCL/parallel_141: two get_theater_movie_releases calls — New York IMAX over 7 days, Los Angeles 2D over 14 days. Tightened: the prompt explicitly names each format, so format is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 789 OUT442

✓

PlotTwoSineWavesBFCL/parallel_146: two plot_sine_wave calls with all five parameters supplied by the prompt.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 895 OUT390

✓

SortFourListsAscendingAndDescendingBFCL/parallel_137: eight array_sort calls — four lists, each sorted ascending AND descending (list element order preserved as given).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 844 OUT1 057

✓

UpdateTwoCustomersNameAndEmailBFCL/parallel_142: two update_user_info calls — users 12345 and 67890, both set name "John" and email "example@.com". update_info is a nested object. Tightened: the prompt names the 'CustomerInfo' database, so database == "CustomerInfo" is required (also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 880 OUT419

Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 3) 10 passed1 failed

90.9%

IN88 034 OUT26 746

✓

AreaUnderCurveForTwoPolynomialsBFCL/parallel_150: two mathematics_calculate_area_under_curve calls — polynomial [3,-2,1] over [-1,2] and [1,0,-1] over [0,3] (coefficient/limit order preserved).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 830 OUT1 081

✓

BinomialProbabilitiesThreeGamesBFCL/parallel_160: three prob_dist_binomial calls — (20,5), (50,15), (100,30), each with p == 0.3 (prompt states a 0.3 win probability). The prompt explicitly names 'prob_dist.binomial'; the colliding calculate_binomial_probability tool is kept in a separate script (ToolCalling-Parallel-D2).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 001 OUT19 592

✓

CalculateBmiSystemFourPeopleBFCL/parallel_154: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt says "use the metric system for all calculations", so system == "metric" is required (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 128 OUT868

✓

DiceRollProbabilitiesThreeScenariosBFCL/parallel_159: three probability_dice_roll calls — roll 4 three times and roll 2 twice on a six-sided die (die_sides default 6, accepts omission), then roll 7 twice on an eight-sided die (die_sides == 8 required, non-default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 958 OUT762

✓

GenerateNormalVariatesTwoConfigsBFCL/parallel_158: four random_normalvariate calls — two at (mu 5, sigma 2) and two at (mu 10, sigma 3). Duplicate calls: each configuration must appear exactly twice (multiplicity 2).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 143 OUT1 135

✓

PredictPricesForFourHousesBFCL/parallel_157: four predict_house_price calls — New York (3/2/1500), Los Angeles (4/3/2000), Chicago (2/1/1200), Miami (3/2/1800).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 252 OUT708

✓

RaiseTwoNumbersToPowersBFCL/parallel_152: two math_power calls — 3^5 and 2^3, no modulus (mod omitted / null).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 684 OUT365

✓

RunLinearRegressionWithAndWithoutStandardizationBFCL/parallel_155: two run_linear_regression calls — same predictors [Age, Income, Education] and target "Spending Score", first without standardization then with. standardize is explicitly requested (false then true), so it is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 896 OUT447

✓

TrainRandomForestFourConfigsBFCL/parallel_156: four random_forest_train calls on "data_random_forest" — (100,10), (200,20), (300,30), (400,40). The prompt explicitly names this function, disambiguating it from train_random_forest_classifier visible in this script.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 310 OUT648

✓

TrainRfClassifierTwoConfigsBFCL/parallel_153: two train_random_forest_classifier calls — dataset1 (depth 10, 100 trees) and dataset2 (depth 20, 200 trees). The prompt explicitly names this function, disambiguating it from the other random-forest training tool (random_forest_train) visible in this script.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 874 OUT426

×

TriangleAreasInDifferentUnitsBFCL/parallel_151: three geometry_area_triangle calls in different units — (15,20) square meters, (25,30) square feet, (35,40) square inches. First unit is the default (accepts omission / square meters); feet and inches are non-default and required.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: match0 == 1 Values: match0 = 0

Failed

IN7 958 OUT714

Tool Calling - Parallel Calls: Finance, Stats & Legal 13 passed

100.0%

IN110 764 OUT39 614

✓

BuildTwoDecisionTreeClassifiersBFCL/parallel_12: two model_DecisionTreeClassifier calls — gini/depth 5 and entropy/depth 10, both random_state 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 440 OUT353

✓

CalculateFutureValueForThreeTermsBFCL/parallel_18: three financials_calculate_future_value calls — years 1, 5, 10 (same 1000 @ 0.05).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 602 OUT756

✓

CalculatePresentValueForThreeTermsBFCL/parallel_14: three calculate_present_value calls — years 10, 20, 30 (same payment 1000, rate 0.05).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 599 OUT1 077

✓

CapitalGainsTaxForTwoStatesBFCL/parallel_15: two calculate_capital_gains_tax calls — CA (15000/25000) and FL (20000/50000).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 492 OUT4 043

✓

ComputePythagorasForTwoTrianglesBFCL/parallel_10: two math_pythagoras calls — sides (3,4) and (5,12).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 383 OUT363

✓

ConfidenceIntervalForTwoSampleSizesBFCL/parallel_13: two confidence_interval_calculate calls — sample sizes 50 and 150. Tightened per methodology: the prompt explicitly requests "95% confidence interval", so confidence_level == 0.95 is required (schema default is 0.9, but the prompt states the value).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 495 OUT1 318

✓

CourtCaseStatusAndTrialDatesBFCL/parallel_22: four court_info_get_case_status calls — cases XY1234 (LA) and GH5678 (Orange County), each queried for status (details may be omitted via the 'status' default) and for trial_date.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 803 OUT832

✓

GetStockPriceAndVolumeForThreeCompaniesBFCL/parallel_17: three get_stock_data calls — AAPL, GOOG/GOOGL, MSFT; each with data_points {price, volume} (unordered).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 534 OUT685

✓

LoanEligibilityForTwoInstitutionsBFCL/parallel_20: two loan_eligibility_check calls — HSBC (500k/100k) and Wells Fargo (700k/120k).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 507 OUT556

✓

MortgagePaymentForThreeLoanTermsBFCL/parallel_19: three calculate_mortgage_payment calls — loan terms 15, 20, 30 (400000 @ 0.04).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 627 OUT28 207

✓

PredictHousePriceInTwoCitiesBFCL/parallel_11: two ml_predict_house_price calls — New York 3000 sq ft, Los Angeles 4000 sq ft.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 414 OUT411

✓

ReturnOnInvestmentGainAndLossBFCL/parallel_16: two calculate_return_on_investment calls — gain 500 on 2000, loss -1000 on 5000.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 445 OUT424

✓

SearchMoneyLaunderingConvictionsBFCL/parallel_21: two law_crimes_search calls — money laundering in San Francisco 2019 and Texas 2018.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 423 OUT589

Tool Calling - Parallel Calls: Legal, History & Environment 12 passed1 failed1 skipped

92.3%

IN122 477 OUT12 349

✓

BookTwoConnectingFlightsBFCL/parallel_30: two book_flight calls — San Francisco→Tokyo (2022-05-03) and Tokyo→Sydney (2022-05-18).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 057 OUT284

✓

CalculateAlimonyForTwoDurationsBFCL/parallel_23: two alimony_calculator_ca_calculate calls — durations 10 and 20 years (10000/3000 income).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 120 OUT1 418

✓

CalculateWasteForFamilyAndBachelorBFCL/parallel_29: two waste_calculation_calculate calls — LA family (2 adults, 2 children) and New York bachelor (1 single). The population argument is a nested object.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 070 OUT1 180

✓

FetchTwoHistoricalFactsBFCL/parallel_31: two history_fact_fetch calls — Treaty of Paris and Magna Carta. Optional depth/year omitted (ground truth accepts the schema defaults).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 976 OUT917

✓

FindClassicalConcertsWithParkingBFCL/parallel_44: two concert_find_nearby calls — classical concerts near Berlin and Paris next Friday. Tightened per methodology: the prompt explicitly asks for concerts "with available parking", so amenities == ['Parking'] is required (this also matches the schema default, but the prompt states it). Isolated from concert_finder (parallel_43) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 050 OUT1 065

–

FindCourtCasesByNumberAndTypeBFCL/parallel_26: four court_case_find calls — the cross product of case numbers {67813, 71249} and types {Civil, Criminal}; case_number is a single-element array; Civil may be omitted via its default. FLAGGED as borderline (see report): the prompt "case numbers '67813','71249' for type 'Civil' and 'Criminal'" is ambiguous about the pairing; BFCL ground truth expects all four combinations, kept faithful.0 AI queries 0 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Skipped: Ambiguous call decomposition (item flaw, not model behavior): 'case numbers 67813, 71249 for type Civil and Criminal' has no single deterministic reading - two calls (both numbers per type, case_number as a list) is as valid as the BFCL 2x2 = four single-number calls; a competent human would not answer deterministically. Kept for traceability.

Skipped

IN0 OUT0

✓

FindNatureReservesNearTwoCitiesBFCL/parallel_27: two nature_reserve_find_nearby calls — Berkeley (10km, picnic tables + restrooms) and Tokyo (5km, playgrounds + biking trails). Amenity list order is unordered.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 058 OUT514

✓

GetCaseDetailsWithHistoryAndLitigantsBFCL/parallel_24: two law_case_get_details calls — cases 28473 and 64725. Tightened per methodology: the prompt explicitly asks for "their history and details of litigants", so include_history == true and include_litigants == true are required (schema defaults are false).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 040 OUT608

✓

GetEventsForTwoPresidenciesBFCL/parallel_32: two us_history_events_by_presidency calls — Abraham Lincoln and George Washington. Optional start_year/end_year omitted (ground truth accepts the schema defaults).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 971 OUT812

✓

GetPresidentsAndVicePresidentsByYearBFCL/parallel_33: four get_president_and_vp calls — president for 1980 and 2016, vice president for 1975 and 2011.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 326 OUT981

✓

GetReligionInfoForTwoEmpiresBFCL/parallel_35: two ancient_empires_get_religion_info calls — Persian Empire and Mauryan Empire. Tightened per methodology: the prompt explicitly asks for "religious history and influences", so include_influences == true is required (schema default is false).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 011 OUT777

✓

GetTemperatureForTwoCitiesBFCL/parallel_28: two get_current_and_future_temperature calls — Seattle and Los Angeles, next 3 hours.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 983 OUT320

×

LookupLawsuitsForTwoYearsBFCL/parallel_25: two lawsuit_lookup calls — Dara Inc for years 2019 and 2018.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 2

Failed

IN13 761 OUT1 507

✓

TrackChristianityInTwoRegionsBFCL/parallel_34: two religion_history_track calls — Christianity in Egypt and Turkey, 100–1500 A.D.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 054 OUT1 966

Tool Calling - Parallel Calls: Markets & Environment 13 passed

100.0%

IN114 708 OUT11 585

✓

CalculateCagrForTwoInvestmentsBFCL/parallel_176: two calculate_cagr calls — (5000,7000,5) and (8000,12000,3).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 665 OUT769

✓

CalculateEmissionSavingsForSolarAndWindBFCL/parallel_186: two calculate_emission_savings calls — solar(12 months) and wind(8 months), Midwest region. Prompt states the Midwest region explicitly -> region required (accept "Midwest"/"Midwest region").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 558 OUT549

✓

CompareFutureValueWithAndWithoutContributionsBFCL/parallel_182: two finance_calculate_future_value calls — (5000,0.07,10, no contribution) and (3000,0.06,10, contribution 200). First scenario states "no additional contributions" -> null or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 725 OUT1 383

✓

ComputeEuclideanNormForThreePointsBFCL/parallel_183: three math_hypot calls — (5,7), (10,15), (20,25). z omitted or 0 (2D points).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 761 OUT1 713

✓

EstimatePopulationsForTwoSpeciesAcrossTwoYearsBFCL/parallel_185: four estimate_population calls — Bengal Tigers/India and African Elephants/Kenya for 2021 and for 2023. NOTE: upstream ground truth's third entry lists year "" (omitted -> default 2020), which is an upstream error: the prompt explicitly asks for 2023 for "both these species". We tighten both 2023 estimates to year == 2023 (fair deterministic answer).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 904 OUT623

✓

FindRootsForTwoQuadraticEquationsBFCL/parallel_184: two algebra_quadratic_roots calls — (3,7,2) and (5,-4,1).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 678 OUT917

✓

GetClosingPricesForTwoCompaniesOnTwoDatesBFCL/parallel_178: four get_stock_price (by-date variant) calls — the 2x2 cross product of {Microsoft, Apple} x {2022-01-01, 2022-02-01} on NASDAQ. Upstream ground truth's first entry loosely lists company_name as [Microsoft, Apple]; the deterministic human answer is the four distinct (company, date) combinations, which we require. exchange NASDAQ (default) may be omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 894 OUT748

✓

GetDetailedAirQualityForThreeDaysBFCL/parallel_187: three get_air_quality calls — New York City, detail true, for 2023-05-05, 05-04, 05-03. Prompt asks for PM2.5/PM10/ozone/sources -> detail == true; "today/yesterday/day before" with today = May 5, 2023.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 764 OUT741

✓

GetFourPriceTypesForTwoCompaniesBFCL/parallel_180: eight stock_price calls — {Microsoft, Apple} x {Open, Close, High, Low}, days 30. Prompt explicitly lists all four data types, so data_type must be set explicitly (required) to distinguish the eight calls; omission (default 'Close') cannot represent the four requested types.1 AI query 8 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Passed

IN9 455 OUT975

✓

GetPricePerOunceForFourMetalsBFCL/parallel_177: four get_metal_price calls — gold, silver, platinum, palladium, measure 'ounce'. Prompt asks "price per ounce" -> measure == "ounce" required.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 785 OUT592

✓

GetStockPricesForFourCompaniesAndDurationsBFCL/parallel_181: four get_stock_prices calls — Apple(1 week), Microsoft(2 weeks), Amazon(3 weeks), Tesla(1 month). Each 'companies' argument is a single-element list.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 834 OUT455

✓

GetTrafficInfoForThreeLegsWithModesBFCL/parallel_188: three get_traffic_info calls — NY->LA (driving), LA->SF (bicycling), SF->NY (transit). First leg explicitly says "driving" -> mode == "driving" required (tightened; also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 759 OUT471

✓

ProjectFutureValueForThreeStocksBFCL/parallel_175: three portfolio_future_value calls — AAPL(5000,0.07,5), MSFT(8000,0.06,7), AMZN(10000,0.08,10).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 926 OUT1 649

Tool Calling - Parallel Calls: Mathematics & Physics (set 1) 12 passed1 failed

92.3%

IN104 061 OUT9 113

✓

AreaOfTwoRightTrianglesInMetersBFCL/parallel_68: two calculate_area calls — right triangles base/height 12x15 and 18x24 in meters. unit required (tool default is 'cm'; the prompt states meters/square meters).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 990 OUT501

✓

CheckPlayerStatusAcrossThreeSeasonsBFCL/parallel_49: three player_status_check calls — player 3142 in RocketLeague for seasons 2017/2018/2019. Seasons are explicitly given, so each is required (default 2023 not applicable).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 071 OUT644

✓

CircumferenceOfFourCirclesInCmBFCL/parallel_65: four calculate_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt gives every radius in cm (e.g. "5cm") and the tool default unit is 'm', so a correct call must pass unit == 'cm'/'centimeter' (ground truth requires it on the first call and merely allows omission on the rest; we require it consistently since it is stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 213 OUT564

×

ConvertEuroAndPoundsToDollarsBFCL/parallel_59: two get_conversion_cost calls — 15000 Euro->dollars and 200 pounds->dollars.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: match1 == 1 Values: match1 = 0

Failed

IN7 901 OUT1 156

✓

FinalVelocityForFourDropsBFCL/parallel_81: four calculate_final_velocity calls — heights 10/20/15/25, initial_velocity 0 (stated), gravity 9.81 (default, may be omitted).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 398 OUT1 165

✓

FindItalianAndJapaneseRestaurantsBFCL/parallel_52: two restaurant_finder calls — Italian in New York (preferences ['Vegetarian']) and Japanese in Los Angeles (delivery; ['Delivery'] or omitted since default is ['Delivery']).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 896 OUT687

✓

FindRootsOfTwoQuadraticsAlgebraBFCL/parallel_62: two algebra_quadratic_roots calls — 3x^2+4x+2 and 5x^2-7x+3.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 063 OUT789

✓

FindSoupAndVeganSaladRecipesBFCL/parallel_51: two recipe_search_find calls — Chicken Noodle Soup (diet omitted/default 'Keto') and a Vegan Salad (diet == 'Vegan', dish name accepts several phrasings).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 831 OUT529

✓

GcdOfJohnAndMaryNumbersBFCL/parallel_80: two math_gcd calls — gcd(36,48) and gcd(60,96). Same tool as parallel_76.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 946 OUT427

✓

GcdOfTwoPairsBFCL/parallel_76: two math_gcd calls — gcd(45,60) and gcd(81,27).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 878 OUT313

✓

GetPokemonMovesAndCheckSolarBeamBFCL/parallel_48: two PokemonGO_get_moves calls — Pikachu's move list (move omitted/default 'Run') and a check whether Bulbasaur can learn 'Solar Beam'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 868 OUT820

✓

PrimeFactorsFormattedAndArrayBFCL/parallel_74: four get_prime_factors calls — numbers 4567 and 7890, each formatted true and false (user wants "a formatted string as well as an array").1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 160 OUT1 198

✓

SaveGameProgressEasyAndHardBFCL/parallel_50: two game_save_progress calls — stage 7 easy and stage 3 hard; level omitted/default 'user'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 846 OUT320

Tool Calling - Parallel Calls: Mathematics & Physics (set 2) 13 passed

100.0%

IN107 949 OUT11 555

✓

AreaOfThreeCirclesInMetersBFCL/parallel_66: three geometry_area_circle calls — radii 5/10/15 in meters (default is meters, so units may be omitted).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 335 OUT640

✓

BookTwoHotelRoomsBFCL/parallel_57: two hotel_booking_book calls — Sheraton (NY, 2 adults/1 child) and Marriott (LA, 1 adult/2 children).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 383 OUT468

✓

CheckAvailabilityForTwoHotelsBFCL/parallel_56: two hotel_booking_check_availability calls — Queens Hotel (Berlin) and Royal Hotel (Paris), each with check-in/out dates (several date formats accepted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 315 OUT584

✓

CircumferenceOfFourCirclesInMetersBFCL/parallel_70: four geometry_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt asks for output in meters and gives radii in m, but the tool default unit is 'cm', so a correct call must pass units == 'm'/'meters' (ground truth requires it on the first call and only allows omission on the rest; we require it consistently since it is explicitly stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 515 OUT685

✓

DerivativeOfPolynomialAndItsDerivativeBFCL/parallel_71: two calculate_derivative calls — 3x^3-2x^2+5x-7 at x=4, then its derivative 9x^2-4x+5 at x=2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 230 OUT1 125

✓

FinalVelocityForThreeDragRaceRoundsBFCL/parallel_83: three final_velocity calls — (v0=0,a=5,t=10), (v0=10,a=7,t=8), (v0=20,a=4,t=12).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 512 OUT496

✓

FindGroceryStoresByCriteriaBFCL/parallel_55: two grocery_store_find_by_criteria calls — NYC ['24 hours'] and San Diego ['Home Delivery'].1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 160 OUT484

✓

GetTwoCookingRecipesBFCL/parallel_53: two get_cooking_recipe calls — Lasagne Bolognese for 4 and Caesar Salad for 2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 166 OUT1 075

✓

GetTwoExchangeRatesBFCL/parallel_58: two get_exchange_rate calls — USD->AUD and USD->CAD.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 130 OUT323

✓

HighestCommonFactorOfFourPairsBFCL/parallel_77: four math_hcf calls — (45,60), (90,120), (36,48), (72,96). FLAGGED (see final report): the prompt is phrased as chaining ("use that result to find the HCF with...") but the ground truth treats it as four INDEPENDENT parallel calls on the literal pairs; a human might interpret it as sequential composition. Kept BFCL-faithful (four independent literal-pair calls) since parallel semantics preclude feeding one call's result into another.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 549 OUT3 495

✓

OrderSameItemsFromTwoWholeFoodsLocationsBFCL/parallel_54: two whole_foods_order calls — same two items (order-independent) at downtown and uptown, size large.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 212 OUT370

✓

PrimeFactorizeAsDictionaryBFCL/parallel_79: two prime_factorize calls — 4567 and 7890, return_type 'dictionary' (explicitly requested).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 172 OUT1 192

✓

SolveTwoQuadraticEquationsBFCL/parallel_63: two solve_quadratic_equation calls — (5,6,1) and (3,2,1). Prompt names this function explicitly.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 270 OUT618

Tool Calling - Parallel Calls: Mathematics & Physics (set 3) 12 passed

100.0%

IN98 273 OUT8 588

✓

AreaOfTwoCirclesInMetersBFCL/parallel_67: two geometry_calculate_area_circle calls — radii 5 and 10 in meters. unit required (tool default is 'cm'; the prompt states meters).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 019 OUT577

✓

AreaOfTwoTrianglesBFCL/parallel_69: two calculate_triangle_area calls — base/height 10x5 and 8x6.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 079 OUT498

✓

DerivativeOfTwoFunctionsAtValuesBFCL/parallel_73: two calculus_derivative calls — 3x^2+2x-1 at 5 (variable x, default) and 4y^3-3y^2+2y-1 at 3 (variable y).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 124 OUT630

✓

DisplacementForThreeCarsBFCL/parallel_84: three calculate_displacement calls — (15,7,accel 3.5), (20,10,accel 2.0), (25,8,no acceleration). The third car travels "without any acceleration" so acceleration == 0 (also the default; omission accepted).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 340 OUT813

✓

EuclideanNormForThreePointsBFCL/parallel_61: three math_hypot calls — (3,4) and (6,8) with z omitted/0, and (9,12,15).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 254 OUT586

✓

FactorialOfThreeNumbersBFCL/parallel_60: three math_factorial calls — 5, 7, 9.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 081 OUT401

✓

FinalSpeedForTwoFreeFallObjectsBFCL/parallel_85: two calculate_final_speed calls — initial_speed 0 for 10 s and initial_speed 5 for 7 s; gravity -9.81 (default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 147 OUT1 098

✓

GcdViaNumberTheoryBFCL/parallel_78: two number_theory_gcd calls — (45,60) and (81,63). Prompt names this function explicitly.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 022 OUT410

✓

IntegrateTwoFunctionsWithTwoMethodsBFCL/parallel_72: four integrate calls — x**3 on [2,5] and 2x**2+3x-1 on [-1,3], each with trapezoid (default, may be omitted) and simpson (explicit).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 468 OUT1 439

✓

PrimeFactorsOfThreeNumbersBFCL/parallel_75: three number_analysis_prime_factors calls — 45, 100, 150.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 087 OUT449

✓

SolveFourQuadraticsWithRootTypeBFCL/parallel_64: four solve_quadratic calls — (2,5,3) all roots, (1,-3,2) real, (4,-7,3) all roots, (1,2,1) real. 'all' is the default so it may be omitted; 'real' is explicitly requested and required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 560 OUT1 040

✓

VelocityForTwoCyclingDaysBFCL/parallel_82: two calculate_velocity calls — 120 km in 5 h and 150 km in 6 h; unit km/h (default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 092 OUT647

Tool Calling - Parallel Calls: Mixed Domains 8 passed2 failed

80.0%

IN73 380 OUT5 603

✓

CalculateBmiForTwoPeopleBFCL/parallel_4: two calculate_bmi calls — 6 ft / 80 kg and 5.6 ft / 60 kg.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 278 OUT491

✓

CalculateEmForceForTwoTimeIntervalsBFCL/parallel_1: two calculate_em_force calls — same field/area, d_time 4 then 10.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 299 OUT524

✓

CalculateFactorialForThreeNumbersBFCL/parallel_7: three math_factorial calls — 5, 10, 15.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 338 OUT385

✓

CalculateResistanceForCopperAndAluminumBFCL/parallel_2: two calculate_resistance calls — copper then aluminum, same length/area. Tightened per methodology: the prompt explicitly names "resistivity of copper", so the first call must pass resistivity == "copper" (ground truth also accepts omission via the 'copper' default, but an explicitly-stated value must be supplied). Aluminum is non-default → required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 286 OUT702

✓

CalculateSalesTaxForThreeCitiesBFCL/parallel_6: three calculate_sales_tax calls — Chicago/IL $30.45, Sacramento/CA $52.33, Portland/OR $11.23.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 470 OUT821

✓

FetchPopulationForCitiesStateAndCountryBFCL/parallel_8: four database_us_census_get_population calls — NYC (city), LA (city), Alaska (state), USA (country); year defaults to 2000 (may be omitted). Requires area-type inference.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 554 OUT567

×

FindMovieShowingsForTwoMoviesBFCL/parallel_9: two find_movie_showing calls near San Diego — Tenet at 5 pm and No Time To Die at 7:30 pm.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 1 expectedCount = 2

Failed

IN7 160 OUT792

×

GetSequenceAnd3DForThreeProteinsBFCL/parallel_3: three protein_info_get_sequence_and_3D calls — HbA1c, normal + rat hemoglobin. Tightened per methodology: the prompt explicitly asks for "their 3D models", so model_3d == true is required (ground truth also accepts omission via the default of true).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: match0 == 1 Values: match0 = 0

Failed

IN7 418 OUT495

✓

ListShowRatingsOnNetflixAndHuluBFCL/parallel_5: two streaming_services calls — Netflix['Friends'] and Hulu['The Office','Stranger Things'] (Hulu show order unordered), both sorted by rating. Must group shows per service.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 315 OUT535

✓

PlayTwoArtistsForGivenDurationsBFCL/parallel_0: two spotify_play calls — Taylor Swift for 20 min and Maroon 5 for 15 min.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 262 OUT291

Tool Calling - Parallel Calls: Physical & Life Sciences (set 1) 11 passed1 failed

91.7%

IN102 115 OUT32 256

✓

ElectricFieldForTwoChargesBFCL/parallel_94: two calculate_electric_field calls — charge 5/distance 2 and charge 3/distance 4. "Conducted in a vacuum" = the default permitivity → may be omitted or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 410 OUT727

✓

ElectromagneticForceInTwoMediaBFCL/parallel_96: two electromagnetic_force calls — same charges 5,10 and distance 2, differing by medium_permittivity. Vacuum call (8.854e-12) equals the default → may be omitted or 8.854e-12; the second medium (5e-12) is non-default → required.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 595 OUT3 786

✓

EstimateTravelTimeForThreeRoutesBFCL/parallel_93: three route_estimate_time calls — (1) NYC->Miami via Philadelphia/Washington D.C./Atlanta, (2) NYC->Miami skipping Atlanta (via Philadelphia/Washington D.C.), (3) Philadelphia-> Miami via Washington D.C. Stop orderings from ground truth accepted. FLAG parallel_93: BORDERLINE. Ground truth allows many stop-list variants, some including the start city itself as a stop; a competent human's stop set is not fully deterministic. Kept BFCL-faithful by accepting the natural stop orderings (start city excluded from stops).1 AI query 3 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Passed

IN8 558 OUT889

✓

FinalSpeedForTwoDroppedObjectsBFCL/parallel_88: two calculate_final_speed calls — v0 0/height 10 and v0 5/height 20. gravity stated as 9.8 (the default) so it may be omitted or 9.8 per ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 416 OUT522

✓

FinalVelocityForTwoCarsBFCL/parallel_87: two calculate_final_velocity calls — Car A (v0 0, a 6, t 10), Car B (v0 20, a 4, t 15).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 433 OUT499

✓

FinalVelocityFromDistanceForTwoObjectsBFCL/parallel_86: two kinematics_final_velocity_from_distance calls — accel 5/dist 100 and accel 10/dist 200. "Both objects start from rest" → initial_velocity may be omitted or 0 (default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 441 OUT634

✓

FindVeganRestaurantsInThreeCitiesBFCL/parallel_91: three vegan_restaurant_find_nearby calls — LA, SF, Seattle, each with operating_hours 22 (prompt: "open until at least 22:00", a non-default value → required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 485 OUT546

✓

GenerateFourTravelItinerariesBFCL/parallel_90: four travel_itinerary_generator calls — Tokyo/7/200/urban, Paris/10/150/history, Sydney/5/100/nature, Rome/12/180/culture. Tokyo's "urban" equals the default → omit or "urban".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 787 OUT675

×

GetDirectionsForRoundTripBFCL/parallel_89: three get_directions calls — SF->Palo Alto (fastest), Palo Alto->Golden Gate Bridge (scenic), Golden Gate Bridge->SF (fastest). route_type explicitly stated each time.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: match0 == 1 Values: match0 = 0

Failed

IN8 524 OUT2 746

✓

MagneticFieldForTwoLoopsBFCL/parallel_95: two calculate_magnetic_field calls — current 10/radius 0.5 and current 15/radius 1.0. "Same as in free space" = default permeability → may be omitted or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 516 OUT935

✓

ResonantFrequencyForTwoCircuitsBFCL/parallel_97: two calculate_resonant_frequency calls — L 0.005/C 1e-07/round 3 and L 0.007/C 2e-07/round 4. Rounding is stated explicitly → required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 480 OUT18 831

✓

ShortestDrivingDistanceRoundTripBFCL/parallel_92: three get_shortest_driving_distance calls — NYC->LA, LA->Miami, Miami->NYC, all in miles (prompt explicitly "in miles" → non-default unit required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 470 OUT1 466

Tool Calling - Parallel Calls: Physical & Life Sciences (set 2) 13 passed

100.0%

IN111 803 OUT16 741

✓

AbsolutePressureAtTwoAltitudesBFCL/parallel_102: two calc_absolute_pressure calls — gauge 2.5 with atm 1 (default; may be omitted) and gauge 2.5 with atm 0.85 (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 519 OUT1 408

✓

BoilingMeltingPointsAcrossSeaLevelsBFCL/parallel_100: four get_boiling_melting_points calls — water@0, iron@1000, water@1000, iron@0. FLAG parallel_100: BORDERLINE. The prompt ("water and iron at sea levels of 0 and 1000 meters respectively") reads most naturally as TWO calls (water@0, iron@1000), but the BFCL ground truth is the full 2x2 cross product (FOUR calls). Kept BFCL-faithful to the ground truth; a competent human could reasonably answer with two calls, so this item's expected call count is not fully deterministic.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 405 OUT1 793

✓

CellInfoDetailedAndBriefBFCL/parallel_108: two biology_get_cell_info calls — neuron with detailed=true (prompt: "detailed description"), muscle with detailed=false ("less detailed", the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 408 OUT2 368

✓

DensityForTwoSubstancesBFCL/parallel_101: two calculate_density calls (mass/volume schema) — 10 kg / 2 m³ and 15 kg / 3 m³. unit "kg/m³" equals the default → may be omitted or "kg/m³".1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 536 OUT451

✓

ElectricFieldStrengthAtSeveralDistancesBFCL/parallel_98: four calculate_electric_field_strength calls — charge 2, distances 0.5/1/2 in vacuum (default medium), plus distance 1 in air. The distance-1 vacuum and distance-1 air calls are distinguished by medium. Vacuum equals the default → may be omitted or "vacuum"; "air" is required.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 831 OUT1 098

✓

EntropyChangeForTwoConditionsBFCL/parallel_103: two entropy_change_calculate calls — substance A, mass 2, temps 25->75 and 10->50. pressure 1 equals the default → may be omitted or 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 638 OUT879

✓

FetchDnaSequenceThreeWaysBFCL/parallel_106: three fetch_DNA_sequence calls for id XYZ123 — default fasta/0, then genbank format, then 500 bp upstream. Defaults (fasta, upstream 0) may be omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 613 OUT414

✓

FunctionLookupForTwoMoleculesBFCL/parallel_110: two cell_biology_function_lookup calls — ATP in mitochondria and DNA in nucleus, both with specific_function=true (prompt asks for a specific function within the organelle).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 469 OUT630

✓

HeatCapacityForThreeConditionsBFCL/parallel_105: three calc_heat_capacity calls — 300K/2.5m³, 350K/2.5m³, 300K/1.5m³, all for air. "air" equals the default gas → may be omitted or "air".1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 717 OUT2 973

✓

MolecularWeightForTwoCompoundsBFCL/parallel_111: two calculate_molecular_weight calls — Glucose (C6H12O6) and Sucrose (C12H22O11), both in grams/mole (prompt explicitly requests that unit; ground truth also accepts "g/mol").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 470 OUT624

✓

PhaseChangeEnergyForWaterBFCL/parallel_99: four thermo_calculate_energy calls — 500 g water, phase melting/freezing/ vaporization/condensation. Substance "water" equals the default → may be omitted or "water".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 902 OUT1 516

✓

ProteinSequencesForTwoGenesTwoSpeciesBFCL/parallel_107: four get_protein_sequence calls — BRCA1 and BRCA2 in Homo sapiens (default species, may be omitted) and in Pan troglodytes (non-default → required).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 732 OUT1 021

✓

ProteinsInThreeCompartmentsBFCL/parallel_109: three cellbio_get_proteins calls — nucleus/mitochondria/cytoplasm, each with include_description=true (prompt asks for "a brief description of each protein").1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 563 OUT1 566

Tool Calling - Parallel Calls: Physical & Life Sciences (set 3) 13 passed

100.0%

IN118 188 OUT12 251

✓

AnalyzeDnaForThreeMutationTypesBFCL/parallel_114: six analyze_dna_sequence calls — two sequences ("AGCTTAGCTA" and "AGCTTAGGCTA") against reference "AGCTTAGCTA", each checked for insertion (default, may be omitted), deletion, and substitution mutations.1 AI query 6 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Passed

IN13 733 OUT2 208

✓

DiabetesPredictionForFourPeopleBFCL/parallel_113: four diabetes_prediction calls for four people with distinct weight/height/activity.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 962 OUT670

✓

EntropyChangeIsothermalAndNotBFCL/parallel_104: two calculate_entropy_change calls — same 300->350 K and heat_capacity 4.18, once isothermal (the default true, may be omitted) and once not isothermal (false, required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 651 OUT1 361

✓

ForestGrowthForTwoRegionsBFCL/parallel_120: two forest_growth_forecast calls — Amazon Rainforest over 10 years and Boreal Forests of Canada over 20 years, excluding human impact (false = the default, may be omitted).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 515 OUT525

✓

GenerateFourDnaSequencesBFCL/parallel_123: four generate_DNA_sequence calls — length 500 with a single-nucleotide preference of A, T, C, G respectively.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 822 OUT577

✓

GeneticSimilarityInTwoFormatsBFCL/parallel_115: four genetics_calculate_similarity calls — human vs chimpanzee and human vs gorilla, each in percentage (default, may be omitted) and fraction. species1 accepts "human"/"Human".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 843 OUT1 432

✓

GenotypeFrequenciesForThreeGenotypesBFCL/parallel_116: three calculate_genotype_frequency calls — allele frequency 0.7, genotypes AA/Aa/aa.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 720 OUT906

✓

IdentifyFourBirdsBFCL/parallel_119: four identify_bird calls — small blue/forest (default size, may be omitted), large black/lake, medium brown/desert, large green/tropical rainforest.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 943 OUT1 088

✓

MutationTypeForTwoSnpsBFCL/parallel_112: two mutation_type_find calls — rs123456 in Homo sapiens (default species, may be omitted) and rs7891011 in Canis lupus familiaris (non-default → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 537 OUT590

✓

PopulationDensityForChinaTwoYearsBFCL/parallel_117: two calculate_density calls (country/population schema) — China in 2000 (pop 1.267 billion) and 2010 (pop 1.341 billion), land area 9.597 million km² both times.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 600 OUT1 087

✓

PrecipitationStatsForThreeTimeFramesBFCL/parallel_118: three ecology_data_precipitation_stats calls — Amazon rainforest for six_months, year, five_years.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 610 OUT533

✓

TurtlePopulationForTwoYearsBFCL/parallel_121: two ecology_get_turtle_population calls — Galapagos Islands in 2015 and 2020, both with species info requested (species=true; years stated explicitly → required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 530 OUT488

✓

VehicleEmissionsForThreeVehiclesBFCL/parallel_122: three calculate_vehicle_emission calls — 15000 miles each: gas with default factor 1.4 (may be omitted), diesel with factor 2.7, EV with factor 0.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 722 OUT786

Tool Calling - Parallel Calls: Services & Lookups 11 passed1 failed

91.7%

IN98 389 OUT6 251

✓

CalculateShortestAndScenicDistancesForTwoLegsBFCL/parallel_190: four calculate_shortest_distance calls — {NYC->LA, LA->Miami} x {Shortest, Scenic}.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 368 OUT838

✓

FetchNewsForTwoTopicsInEuropeBFCL/parallel_192: two get_news calls — Climate Change and Artificial Intelligence, 5 articles each, region Europe. Region "Europe" explicitly requested -> required (accept "Europe"/"EU").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 049 OUT273

✓

FindLibrariesWithFacilitiesInThreeCitiesBFCL/parallel_191: three public_library_find_nearby calls — NY[Reading Room,Fiction], LA[Wi-Fi,Children Section], Chicago[Cafe,Reading Room]. Facilities compared as an unordered pair.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 267 OUT550

✓

FindParksWithAmenitiesInThreeCitiesBFCL/parallel_189: three parks_find_nearby calls — NY[Tennis Court,Picnic Area], LA[Playground,Running Track], Chicago[Tennis Court,Playground]. Amenities compared as an unordered pair.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 248 OUT554

×

FindUpcomingEventsInThreeCitiesBFCL/parallel_194: three event_finder_find_upcoming calls — jazz/LA(14d), rock/Chicago(10d), classical/Boston(7d). All day windows explicitly stated -> days_ahead required (7 for Boston, which also equals the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: match0 == 1 Values: match0 = 0

Failed

IN8 262 OUT570

✓

GetCurrentWeatherForFourCitiesBFCL/parallel_199: four get_current_weather calls — New York, Los Angeles, London, Tokyo, with both temperature and humidity included. Prompt explicitly asks to include both -> flags == true (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 343 OUT611

✓

GetDetailedBriefsForTwoMoviesBFCL/parallel_195: two movie_details_brief calls — Inception and The Dark Knight, extra_info true. Prompt asks for Director/Cast/Awards -> extra_info == true required.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 981 OUT259

✓

GetLawsuitDetailsForTwoCasesBFCL/parallel_196: two get_lawsuit_details calls — case 12345 (NY Supreme Court, with verdict) and case 67890 (LA Superior Court, no verdict). "would like the verdict" -> true; "do not need" -> null/false.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 168 OUT704

✓

GetLawsuitInfoForTwoCasesByYearBFCL/parallel_197: two lawsuit_info calls — 12345ABC (2018, New York) and 67890XYZ (2019, California). Years are explicitly stated -> required. First location "New York" may be omitted (matches the default).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 167 OUT468

✓

GetStockPricesForTwoCompaniesOverDaysBFCL/parallel_179: two get_stock_price (by-days variant) calls — Apple on NASDAQ for 10 days, Microsoft on NYSE for 15 days. Both exchanges are explicitly named -> required (NASDAQ / NYSE). The tool's 'company' field is a ticker symbol; company names are accepted as synonyms.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 050 OUT526

✓

SearchLawsuitsForTwoEntitiesInCaliforniaBFCL/parallel_198: two lawsuit_search calls — Google/Santa Clara and Facebook/San Mateo, state California. Prompt states "both in the state of California" -> state required (accept "California"/"CA").1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 087 OUT372

✓

SendTwoEmailsWithCcAndBccBFCL/parallel_193: two send_email calls — Project Update to john.doe (cc manager, bcc hr) and Meeting Reminder to jane.doe (cc assistant, bcc hr). Bodies matched exactly.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 399 OUT526

Tool Calling - Parallel Calls: Statistics & Finance 13 passed

100.0%

IN117 775 OUT20 813

✓

CalculateBinomialProbabilitiesForBasketballShotsBFCL/parallel_163: three calc_binomial_prob calls — (10,6,0.6), (10,6,0.5), (15,6,0.5).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 157 OUT1 859

✓

CalculateCompoundInterestForThreePeriodsBFCL/parallel_170: three calculate_compound_interest calls — principal 5000, rate 0.025, time 2/3/5. Prompt states "compounded quarterly, 4 times in a year" -> n == 4 required (tightened; also ground truth).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 164 OUT4 818

✓

CalculateDiscountedCashFlowForTwoBondsBFCL/parallel_169: two calculate_discounted_cash_flow calls — (50,5,0.05) and (60,7,0.04). Prompt explicitly states "face value of the bond is $1000" for both -> face_value == 1000 required (ground truth also accepts omission via the default, but an explicitly-stated value must be supplied).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 012 OUT2 283

✓

CalculateEventProbabilitiesForCardDeckBFCL/parallel_162: three probability_of_event calls — Ace 4/52 as decimal, heart 13/52 as decimal, red card 26/52 as ratio. format_as_ratio: "as a decimal" -> false or omitted; "as a ratio" -> true (required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 124 OUT669

✓

CalculateQuarterlyDividendForThreeScenariosBFCL/parallel_168: three finance_calculate_quarterly_dividend_per_share calls — (5M,2M), (6M,2M), (6M,2.5M).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 288 OUT1 349

✓

CalculateReturnOnEquityForTwoCompaniesBFCL/parallel_171: two calculate_return_on_equity calls — (1M,5M,200k) and (2M,10M, no dividends). Second company "did not pay any dividends" -> dividends_paid null or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 955 OUT1 932

✓

CalculateRoiForThreeStocksBFCL/parallel_174: three calculate_return_on_investment calls — (150,180,20), (200,210,30), (250,300,40). Dividends are explicitly stated per stock -> required.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 096 OUT754

✓

ComputeRSquaredForTwoRegressionModelsBFCL/parallel_167: two linear_regression_get_r_squared calls on the same CSV — swap education/credit_score as dependent variable. Independent-variable order follows the prompt/ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 930 OUT687

✓

PredictFutureValueForTwoInvestmentsBFCL/parallel_172: two finance_predict_future_value calls — (5000,0.05,10) and (7000,0.04,15). Prompt states "compound interest annually" -> compounding_periods_per_year == 1 required (tightened; also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 940 OUT1 260

✓

PredictHousePricesForThreeHousesBFCL/parallel_166: three predict_house_price calls — NY 2500/3/2000, LA 3000/3/2005, Chicago 2000/2/1995. The second house is described as "similar" with only area/year/location changed, so rooms == 3 per ground truth.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 147 OUT1 073

✓

PredictProfitForTwoFundsBFCL/parallel_173: two investment_predictProfit calls — (5000,0.07,5) and (8000,0.05,7).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 916 OUT1 029

✓

RunChiSquaredTestForTwoContingencyTablesBFCL/parallel_164: two chi_squared_test calls — tables [45,55,35,65] and [30,70,50,50], alpha 0.05. Prompt states "significance level of 0.05 for both" -> alpha == 0.05 required (tightened). Table order is fixed.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 900 OUT1 301

✓

RunTTestAtTwoSignificanceLevelsBFCL/parallel_165: two t_test calls — same datasets, alpha 0.05 then 0.01 (both stated -> required).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 146 OUT1 799

Tool Calling - Simple: Arts, Venues & Music (set 1) 8 passed

100.0%

IN50 785 OUT3 614

✓

SimpleConcertDetailsBeyonceBFCL/simple_python_286 (concert.get_details): artist=Beyonce, location=San Diego, date=April 2022.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 358 OUT306

✓

SimpleConcertInfoPopNyBFCL/simple_python_284 (concert_info.get): location=New York, date=next month, genre=Pop.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 333 OUT384

✓

SimpleGenerateMelodyC4BFCL/simple_python_291 (music_generator.generate_melody): key=C, start_note=C4, length=16, tempo=120 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 381 OUT256

✓

SimpleKeySignatureCsharpBFCL/simple_python_298 (music_theory.key_signature): key='C#', scale_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 318 OUT286

✓

SimpleMuseumHoursLouvreBFCL/simple_python_273 (museum.get_hours): name='Louvre Museum', location=Paris, day omitted or Monday.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 325 OUT412

✓

SimplePaintNeededWall30x12BFCL/simple_python_259 (calculate_paint_needed): coverage_rate=400, length=30, height=12.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 381 OUT349

✓

SimpleReligionHistoryChristianity16cBFCL/simple_python_252 (get_religion_history): religion=Christianity, century=16, sort_by=importance, count=10.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 358 OUT1 237

✓

SimpleSculptureInfoPlensaBFCL/simple_python_263 (get_sculpture_info): artist_name='James Plensa', detail=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 331 OUT384

Tool Calling - Simple: Arts, Venues & Music (set 2) 8 passed

100.0%

IN50 547 OUT3 870

✓

SimpleComposeMelodyCFGBFCL/simple_python_292 (compose_melody): progression=[C,F,G], measures=4, instrument=Piano or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 319 OUT416

✓

SimpleConcertFindDetailsWeekndBFCL/simple_python_290 (concert.find_details): artist=The Weeknd, month=December, year=2022 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 297 OUT483

✓

SimpleFindConcertRockChicagoBFCL/simple_python_285 (find_concert): location=Chicago, price=100, genre=Rock.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 307 OUT337

✓

SimpleMuseumInfoMetBFCL/simple_python_274 (museum_info): museum_name=Metropolitan Museum of Art, info_type=opening_hours or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 295 OUT470

✓

SimpleMusicalScaleCsharpBFCL/simple_python_299 (musical_scale): key='C#'/'C sharp', scale_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 287 OUT603

✓

SimplePaintRequirementWallExclusionBFCL/simple_python_260 (paint_requirement.calculate): area {width=20,height=12}, paint_coverage=350, exclusion {type=window, area=15}.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN6 416 OUT464

✓

SimpleReligionHistoryChristianity300To400BFCL/simple_python_254 (get_religion_history): religion=Christianity, start_year=300, end_year=400, event_type=all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 322 OUT628

✓

SimpleSculptureDetailsDavidBFCL/simple_python_264 (sculpture.get_details): artist=Michelangelo, title=David, detail=size.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 304 OUT469

Tool Calling - Simple: Arts, Venues & Music (set 3) 9 passed

100.0%

IN60 197 OUT3 176

✓

SimpleChordProgressionKeyBFCL/simple_python_297 (music.theory.chordProgression): progression=[I,V,vi,IV], bools any.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 693 OUT296

✓

SimpleCompoundInterest10kBFCL/simple_python_269 (calculate_compound_interest): principle=10000, interest_rate=0.05, time=10, compounds_per_year=1 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 725 OUT427

✓

SimpleConcertSearchClassicalLaBFCL/simple_python_287 (concert.search): genre=classical, location=Los Angeles/LA, date='this weekend', price_range=cheap.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 692 OUT273

✓

SimpleCreateMixCMajorBFCL/simple_python_293 (music_composer.create_mix): scale='C Major', note_duration=quarter, track_length=180.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 715 OUT253

✓

SimpleGenerateCircleImageRedBFCL/simple_python_256 (generate_circle_image): radius=50, color=Red, background=white or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 684 OUT184

✓

SimpleMuseumWorkingHoursLouvreBFCL/simple_python_276 (museum_working_hours.get): museum=Louvre, location=Paris, day=Monday or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 670 OUT324

✓

SimpleRetrieveReligionInfoBuddhismBFCL/simple_python_253 (retrieve_religion_info): religion_name=Buddhism, detail_level=full.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 665 OUT749

✓

SimpleSculptureSearchChicago19cBFCL/simple_python_265 (sculpture_search): location=Chicago, time_frame='19th century', material=all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 682 OUT292

✓

SimpleTimeDifferenceSfSydneyBFCL/simple_python_250 (get_time_difference): place1=San Francisco/SF, place2=Sydney.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 671 OUT378

Tool Calling - Simple: Arts, Venues & Music (set 4) 9 passed

100.0%

IN56 930 OUT4 341

✓

SimpleBuildingDimensionsEmpireStateBFCL/simple_python_270 (building.get_dimensions): building_name=Empire State Building, unit=feet.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 320 OUT451

✓

SimpleConcertFindNearbySeattleBFCL/simple_python_289 (concert.find_nearby): location=Seattle, genre=jazz.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 311 OUT306

✓

SimpleCreateChordProgressionCBFCL/simple_python_294 (music_generation.create_chord_progression): key=C, chords=4, progression_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 324 OUT384

✓

SimpleEarliestReferenceJesusBFCL/simple_python_251 (get_earliest_reference): name='Jesus Christ', source='historical records'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 318 OUT907

✓

SimpleFindInstrumentAcousticBFCL/simple_python_280 (find_instrument): budget=1000, type=acoustic, make omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 317 OUT281

✓

SimpleGuitarPriceLesPaulBFCL/simple_python_283 (guitar_price.find): model='Gibson Les Paul', condition=Excellent, location=Chicago.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 334 OUT409

✓

SimpleIdentifyColorSeaGreenBFCL/simple_python_257 (identify_color_rgb): color_name='Sea Green', standard=basic or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 315 OUT517

✓

SimpleMuseumInfoBritishDateBFCL/simple_python_277 (museum_info): museum='The British Museum', date=2023-06-20, information=[opening_hours,ticket_price] (either order).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN6 366 OUT768

✓

SimpleSculptureValueThinkerBFCL/simple_python_266 (get_sculpture_value): sculpture='The Thinker', artist=Rodin.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 325 OUT318

Tool Calling - Simple: Arts, Venues & Music (set 5) 9 passed

100.0%

IN57 659 OUT3 920

✓

SimpleCircleDimensionsRadius5BFCL/simple_python_272 (calculate_circle_dimensions): radius=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 384 OUT276

✓

SimpleDrawRectangleRedBFCL/simple_python_261 (draw_rectangle): width=20, height=10, color=red.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 412 OUT204

✓

SimpleFindFluteYamahaBFCL/simple_python_282 (find_flute): brand=Yamaha, specs=[open hole, C foot, silver headjoint].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 426 OUT319

✓

SimpleGenerateScaleProgressionCBFCL/simple_python_296 (music_generator.generate_scale_progression): key=C, tempo=80, duration=4, scale_type=major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 419 OUT353

✓

SimpleInstrumentDetailsPianoYamahaBFCL/simple_python_278 (get_instrument_details): instrument=piano, manufacturer=Yamaha, features=[price,rating] (either order).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 399 OUT743

✓

SimpleMetTopArtworksBFCL/simple_python_275 (metropolitan_museum.get_top_artworks): number=5, sort_by=popularity or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 410 OUT484

✓

SimpleMixPaintYellowBlueBFCL/simple_python_258 (mix_paint_color): color1=yellow, color2=blue, lightness=60.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 406 OUT185

✓

SimplePapalBiographyInnocentIIIBFCL/simple_python_255 (religious_history.get_papal_biography): papal_name=Innocent III, include_contributions=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 395 OUT939

✓

SimpleSculptureLocatorMichelangeloBFCL/simple_python_268 (sculpture_locator.find_by_artist): artist=Michelangelo, material=Marble, location=Rome.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 408 OUT417

Tool Calling - Simple: Arts, Venues & Music (set 6) 7 passed

100.0%

IN41 884 OUT2 874

✓

SimpleAnalyzeStructureB1004BFCL/simple_python_271 (analyze_structure): building_id=B1004, floors=[2,3,4], mode=dynamic.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 006 OUT399

✓

SimpleConcertBookingEminemBFCL/simple_python_288 (concert_booking.book_ticket): artist=Eminem, city=New York City, num_tickets=2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 966 OUT166

✓

SimpleFindExhibitionNyBFCL/simple_python_267 (find_exhibition): location='New York City, NY', art_form=sculpture, month omitted, user_ratings=high.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 982 OUT1 043

✓

SimpleInstrumentInfoViolin1721BFCL/simple_python_281 (get_instrument_info): name=Violin, maker=Stradivarius, year=1721.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 000 OUT266

✓

SimpleInstrumentPriceStratocasterBFCL/simple_python_279 (instrument_price.get): brand=Fender, model='American Professional II Stratocaster', finish=Rosewood.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 980 OUT373

✓

SimpleModifyPaintingOilBFCL/simple_python_262 (modify_painting): size='12x18', medium=oil, dominant_color=red.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 983 OUT250

✓

SimpleSongLyricsBohemianBFCL/simple_python_295 (get_song_lyrics): song_title='Bohemian Rhapsody', artist_name=Queen, lang=English or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 967 OUT377

Tool Calling - Simple: Food, Travel & Shopping (set 1) 7 passed1 failed

87.5%

IN51 777 OUT3 278

✓

SimpleCookingConvertPoundsToOuncesBFCL/simple_python_365 (cooking_conversion.convert): 2 pounds->ounces of butter.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 466 OUT282

✓

SimpleCurrencyExchangeUsdToCadBFCL/simple_python_388 (currency_exchange.convert): USD->CAD, amount=500.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 466 OUT365

×

SimpleGroceryStoreOrganicHoustonBFCL/simple_python_369 (grocery_store.find_nearby): Houston, categories {Organic,Fruits,Vegetables} any order.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.categories.Count >= 2 Values: args.categories.Count = 1

Failed

IN6 467 OUT379

✓

SimpleHotelBookPetFriendlyManhattanBFCL/simple_python_380 (hotel_booking): Manhattan, single, 3 nights, 2023-03-10, pet_friendly.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN6 534 OUT639

✓

SimpleMapsDistanceEiffelLouvreBFCL/simple_python_394 (maps.get_distance_duration): Eiffel Tower -> Louvre Museum, traffic omitted/false.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 467 OUT351

✓

SimpleRecipeFindByDietMealBFCL/simple_python_353 (find_recipes): diet=gluten-free, meal_type=dinner, ingredients omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 447 OUT488

✓

SimpleRestaurantFinderItalianNYCBFCL/simple_python_361 (restaurant_finder): city=NYC, cuisine=Italian, diet=Gluten-free.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 462 OUT410

✓

SimpleTimeZoneLondon24hBFCL/simple_python_376 (time_zone_converter): city=London, country=UK, display_format=24h.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 468 OUT364

Tool Calling - Simple: Food, Travel & Shopping (set 2) 8 passed1 failed

88.9%

IN63 445 OUT3 122

✓

SimpleBestSushiPlacesTokyoBFCL/simple_python_362 (get_best_sushi_places): city=Tokyo, top=5, review_rate=4.0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 054 OUT421

✓

SimpleCurrencyConverterUsdToGbpBFCL/simple_python_389 (currency_converter): USD->GBP, amount=200.0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 049 OUT402

✓

SimpleCurrentTimeSydneyCityBFCL/simple_python_377 (get_current_time): city=Sydney, country=Australia, format omitted/HH:MM:SS.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 020 OUT258

✓

SimpleDistanceCalcNyToBostonTerrainBFCL/simple_python_397 (distance_calculator.calculate): New York -> Boston, consider_terrain=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 036 OUT168

✓

SimpleHiltonAvailabilityParisBFCL/simple_python_381 (hilton_hotel.check_availability): Paris, 2023-04-04..08, 2 adults, chain Hilton/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 100 OUT389

✓

SimpleRecipeCaloriesBeefLasagnaBFCL/simple_python_355 (recipe_info.get_calories): website=Foodnetwork.com, recipe=Beef Lasagna, meal_time omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 034 OUT579

✓

SimpleRecipeUnitTbspToTspBFCL/simple_python_366 (recipe.unit_conversion): 2 tablespoon->teaspoon, precision omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 041 OUT242

×

SimpleSafewayOrderPaloAltoBFCL/simple_python_370 (safeway.order): Palo Alto, [olive oil, rice], [3,1]. EXPECTED MODEL MISS (kept, do not loosen): "a five pound bag of rice" = quantity 1 (5 is the weight, not the count); models tend to send quantity 5. This is a genuine model error the test correctly catches.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.items[0].ToLower()).AnyOf(["olive oil"])

Failed

IN7 078 OUT375

✓

SimpleVeganSoupRecipeUnder30BFCL/simple_python_354 (get_vegan_recipe): dish_type=soup, cooking_time=30, ingredient_preference omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 033 OUT288

Tool Calling - Simple: Food, Travel & Shopping (set 3) 8 passed

100.0%

IN51 971 OUT2 858

✓

SimpleBookHotelHiltonChicagoBFCL/simple_python_382 (book_hotel): Hilton Hotel, Chicago, single, 2022-12-10, 2 nights.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN6 544 OUT303

✓

SimpleCookingTimeRoastChickenBFCL/simple_python_368 (calculate_cooking_time): weight_kg=1.5, method omitted/roast, temp omitted/180.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 484 OUT569

✓

SimpleCurrencyConvertEurToCadBFCL/simple_python_390 (currency_conversion.convert): 150 EUR->CAD.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 483 OUT318

✓

SimpleCurrentTimeSydneyLocationBFCL/simple_python_379 (get_current_time): location=Sydney, country=Australia, timezone omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 472 OUT401

✓

SimpleParkingLotCentralParkBFCL/simple_python_395 (parking_lot.find_nearest): Central Park NY, radius=2, type omitted/public.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 488 OUT356

✓

SimpleRecipeFinderVeganServes2BFCL/simple_python_356 (recipe_finder.find): servings=2, diet=vegan, prep_time=30.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 508 OUT298

✓

SimpleRestaurantClosestSushiBostonBFCL/simple_python_363 (restaurant_search.find_closest): Boston, Sushi, amenities=[Patio].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 495 OUT298

✓

SimpleWholeFoodsPriceLABFCL/simple_python_371 (whole_foods.check_price): Los Angeles, [tomatoes, lettuce].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 497 OUT315

Tool Calling - Simple: Food, Travel & Shopping (set 4) 7 passed

100.0%

IN41 988 OUT2 797

✓

SimpleBookRoomThePlazaBFCL/simple_python_383 (book_room): The Plaza, single, 2 nights.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 000 OUT180

✓

SimpleConvertCurrencyJpyToUsdBFCL/simple_python_393 (convert_currency): 20000 JPY->USD.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 004 OUT579

✓

SimpleFindRestaurantItalianBrooklynBFCL/simple_python_364 (find_restaurant): Brooklyn, Italian, Gluten-free.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 995 OUT208

✓

SimpleGetRecipeVeganChocCakeBFCL/simple_python_357 (get_recipe): dish_name=(vegan) chocolate cake, diet_preference=vegan.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 987 OUT722

✓

SimpleHospitalLocateDenverBFCL/simple_python_396 (hospital.locate): Denver CO, radius=5, department=Pediatrics.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 002 OUT324

✓

SimpleTimezoneConvertNyToLondonBFCL/simple_python_378 (timezone.convert): 3pm, New York -> London.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 998 OUT538

✓

SimpleWholeFoodsTopBrandsBananasBFCL/simple_python_372 (whole_foods.find_top_brands): product=bananas, number omitted/5, organic=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 002 OUT246

Tool Calling - Simple: Food, Travel & Shopping (set 5) 5 passed1 failed

83.3%

IN35 474 OUT3 024

×

SimpleExchangeRateFeeGbpJpyBFCL/simple_python_391 (get_exchange_rate_with_fee): GBP->JPY, fee=0.02.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.base_currency?.ToLower()).AnyOf(["gbp"])

Failed

IN5 896 OUT387

✓

SimpleHotelBookingBookParisBFCL/simple_python_384 (hotel_booking.book): Paris, 07-10-2022..07-20-2022, 2 adults, 1 child, Standard/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 959 OUT373

✓

SimpleMuseumHoursMetSaturdayBFCL/simple_python_398 (get_museum_hours): museum=Metropolitan Museum of Art/The Met, day=Saturday.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 873 OUT455

✓

SimpleRecipeSearchGfCookieBFCL/simple_python_358 (recipe_search): diet=[gluten free], time_limit=30, dish=cookie.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 890 OUT400

✓

SimpleRestaurantSearchRatedNYCBFCL/simple_python_399 (restaurant_search): NYC, Italian, rating=4, accepts_credit_cards=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 908 OUT582

✓

SimpleWalmartPurchaseSanJoseBFCL/simple_python_373 (walmart.purchase): San Jose, [apples, rice, (bottled) water], pack_size=[1,1,12].1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 948 OUT827

Tool Calling - Simple: Food, Travel & Shopping (set 6) 5 passed

100.0%

IN27 814 OUT2 062

✓

SimpleGameHighestScoringPlayerBFCL/simple_python_350 (get_highest_scoring_player): game=Valorant, season=2022.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 554 OUT371

✓

SimpleHotelBookRoomKingLABFCL/simple_python_385 (hotel_bookings.book_room): LA, king size, 15-10-2023, 2 nights, rooms omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 601 OUT290

✓

SimpleLatestExchangeGbpJpyBFCL/simple_python_392 (latest_exchange_rate): GBP->JPY, amount omitted/1.0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 542 OUT314

✓

SimpleNutritionalInfoAvocadoBFCL/simple_python_374 (grocery_info.nutritional_info): Walmart, avocado, [Protein,Calories,Carbohydrates].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 561 OUT416

✓

SimpleRecipeSearchVegPastaBFCL/simple_python_359 (recipe_search): dietary_restriction=Vegetarian, ingredients=[pasta,cheese], servings=2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 556 OUT671

Tool Calling - Simple: Food, Travel & Shopping (set 7) 4 passed

100.0%

IN20 861 OUT1 191

✓

SimpleBookHotelLuxuryParadiseBFCL/simple_python_386 (book_hotel): Hotel Paradise, Las Vegas, luxury, 05-12-2022, 3 days, view=city.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 264 OUT320

✓

SimpleFindRecipeCaloriesCarbonaraBFCL/simple_python_360 (find_recipe): recipeName=pasta carbonara, maxCalories=500.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 193 OUT210

✓

SimpleGameMultiplayerFinderBFCL/simple_python_351 (multiplayer_game_finder): platform=Windows 10, rating=4.5, genre omitted/Action.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 196 OUT294

✓

SimpleWalmartPriceCheckPumpkinsBFCL/simple_python_375 (walmart.check_price): [pumpkins,eggs], quantities [3,24] or [3,2], store omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 208 OUT367

Tool Calling - Simple: Food, Travel & Shopping (set 8) 3 passed

100.0%

IN14 598 OUT1 584

✓

SimpleFindRecipeVeganBrowniesBFCL/simple_python_367 (find_recipe): dietary_restrictions=vegan, recipe_type=dessert (brownies), time=30.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 848 OUT708

✓

SimpleGameAverageUserScoreBFCL/simple_python_352 (gamespot.getAverageUserScore): game=Zelda BotW, platform=Nintendo Switch/all platforms.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 858 OUT494

✓

SimpleHotelBookingPlazaRangeBFCL/simple_python_387 (hotel_booking): Plaza Hotel, New York NY, 2022-06-01..2022-06-04, rooms omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 892 OUT382

Tool Calling - Simple: Knowledge, Places & History (set 1) 10 passed

100.0%

IN69 289 OUT4 538

✓

SimpleAirQualityChicagoBFCL/simple_python_203 (get_air_quality): location=Chicago, detail=true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 923 OUT313

✓

SimpleEvolutionScientistBFCL/simple_python_242 (get_scientist_for_discovery): discovery=Theory of Evolution.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 914 OUT224

✓

SimpleGermanWarEventsBFCL/simple_python_231 (history.get_key_events): country=Germany/DE, 1871-1945, event_type=[War].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 963 OUT1 254

✓

SimpleKingOfFranceBFCL/simple_python_230 (historic_leader_search): location=France, date=1510, title omitted or King.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 919 OUT247

✓

SimplePandaPopulationBFCL/simple_python_201 (estimate_population): species=panda(s), country=China/CN, year omitted or 2024.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 915 OUT411

✓

SimplePersonalityTraitsTypeBFCL/simple_python_227 (get_personality_traits by type): type=ENFJ, traits=[strengths, weaknesses].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 926 OUT634

✓

SimplePresidentCivilWarBFCL/simple_python_238 (us_history.get_president): event=American Civil War, year in 1861-1865.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 925 OUT478

✓

SimpleRestaurantNearbyBFCL/simple_python_204 (restaurant.find_nearby): location=Seattle[, WA], cuisine=Chinese, max_distance=10.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 936 OUT313

✓

SimpleTordesillasDateBFCL/simple_python_233 (european_history.get_event_date): event=Treaty of Tordesillas, format=YYYY.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 939 OUT310

✓

SimpleTrafficBostonNewYorkBFCL/simple_python_205 (get_traffic_info): start=Boston, end=New York/NYC, mode omitted or driving.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 929 OUT354

Tool Calling - Simple: Knowledge, Places & History (set 2) 9 passed1 failed

90.0%

IN72 900 OUT5 165

✓

SimpleBitcoinNewsBFCL/simple_python_210 (get_news): topic=Bitcoin, quantity=5, region omitted or US.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 890 OUT263

×

SimpleEuropeanWars19cBFCL/simple_python_234 (history_eu.fetch_events): century=19, region=any of N/S/E/W, category=Wars. EXPECTED MODEL MISS (kept, do not loosen): region is an enum (Northern/Southern/Eastern/Western); the prompt says only "European", so any of the four is accepted, but models often send the invalid value "Europe". Failing to respect the enum is a genuine model error the test correctly catches.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN10 846 OUT1 947

✓

SimpleLibraryNearbyBFCL/simple_python_209 (public_library.find_nearby): location=Boston MA/Massachusetts, facilities=[Fiction, Wi-Fi] (any order).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 922 OUT420

✓

SimpleMonarchOfEnglandBFCL/simple_python_232 (monarch.getMonarchOfYear): location=England/ENG, year=1800, fullName=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 907 OUT344

✓

SimpleNeutronDiscovererBFCL/simple_python_243 (get_discoverer): discovery=neutron, detail=true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 886 OUT455

✓

SimpleParksNearbyBFCL/simple_python_206 (parks.find_nearby): location=London[, UK], amenities=[Tennis Court].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 900 OUT381

✓

SimplePersonalityTraitsHobbyBFCL/simple_python_228 (get_personality_traits by hobby): hobby=jogging, trait_count=3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 889 OUT311

✓

SimplePopulationGrowthBFCL/simple_python_221 (population_growth_estimate): location=London, years=5, rate omitted or 1.2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 892 OUT468

✓

SimplePresidentDuringEventBFCL/simple_python_241 (US_President_During_Event): event=Civil War, country omitted or USA.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 885 OUT240

✓

SimpleTreatyLisbonDateBFCL/simple_python_235 (get_event_date): event=Treaty of Lisbon (variants), location omitted or Lisbon[, Portugal].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 883 OUT336

Tool Calling - Simple: Knowledge, Places & History (set 3) 9 passed1 failed

90.0%

IN72 338 OUT3 004

✓

SimpleAppleStockInfoBFCL/simple_python_212 (get_stock_info): company=Apple Inc., detail_level=detailed, market omitted or NASDAQ.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 219 OUT348

✓

SimpleCivilWarStartDateBFCL/simple_python_236 (us_history.get_event_info): event=American Civil War/Civil War, specific_info=Start Date.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 220 OUT241

✓

SimpleFlightBookBFCL/simple_python_213 (flight.book): departure=San Francisco/SF, destination=London, date=2022-04-27 (variants), time=afternoon, direct=true.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 274 OUT230

✓

SimpleGravitationPublicationYearBFCL/simple_python_244 (publication_year.find): author=Isaac Newton, work=Law of Universal Gravitation (variants), location omitted or all.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 230 OUT279

✓

SimpleMovieBriefBFCL/simple_python_215 (movie_details.brief): title=Interstellar, extra_info omitted or false.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 209 OUT656

✓

SimplePresidentIn1861BFCL/simple_python_239 (US_president.in_year): year=1861, full_name omitted or true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 237 OUT402

✓

SimpleRadiumDiscovererBFCL/simple_python_245 (discoverer.get): element_name=radium (quote variants), year omitted or 0, first omitted or true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 199 OUT169

✓

SimpleSendEmailBFCL/simple_python_211 (send_email): to=john.doe@example.com, subject=Meeting, body (opt trailing period), cc/bcc empty.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 276 OUT176

✓

SimpleShortestDistanceBFCL/simple_python_207 (calculate_shortest_distance): start=New York (variants), end=Miami (variants), route_preference=Shortest.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 243 OUT186

×

SimpleUpcomingConcertsBFCL/simple_python_214 (event_finder.find_upcoming): location=New York (variants), genre=Rock, days_ahead=30.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.genre.ToLower() == "rock"

Failed

IN7 231 OUT317

Tool Calling - Simple: Knowledge, Places & History (set 4) 9 passed1 failed

90.0%

IN71 813 OUT4 556

✓

SimpleBmiCalcBFCL/simple_python_222 (calculate_bmi): weight=70, height=180, unit omitted or metric.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 179 OUT372

✓

SimpleCarEmissionsBFCL/simple_python_200 (calculate_emissions): distance=12000, fuel_type=gas, fuel_efficiency omitted or 25.0, efficiency_reduction omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 213 OUT687

✓

SimpleFmriAnalyzeBFCL/simple_python_217 (fMRI.analyze): data_source path, sequence_type=multi-band, smooth=6, voxel_size=2.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 222 OUT304

✓

SimpleGetDirectionsBFCL/simple_python_208 (map_service.get_directions): start=New York/NYC, end=Los Angeles/LA, avoid=[highways, tolls] (any order).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 181 OUT279

✓

SimpleGravityDiscoveryBFCL/simple_python_246 (science_history.get_discovery_details): discovery=Gravity, method_used omitted or default.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 139 OUT633

✓

SimpleGroupDynamicsBFCL/simple_python_223 (group_dynamics.pattern): total=50, extroverts=15, introverts=35.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 199 OUT523

×

SimpleMostFollowedBFCL/simple_python_224 (social_media_analytics.most_followed): topic=psychology, sub_topics=[behaviour, group dynamics], region omitted or all.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.sub_topics[0].ToLower() == "behaviour"

Failed

IN7 177 OUT802

✓

SimpleNeuronCoordinatesBFCL/simple_python_219 (get_neuron_coordinates): neuron_type=GABA, brain_region=All/all part of the brain/entire brain.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 174 OUT415

✓

SimplePresidentIn1940BFCL/simple_python_240 (history_api.get_president_by_year): year=1940, full_term_only omitted/true/false.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 151 OUT252

✓

SimpleSentimentAnalysisBFCL/simple_python_216 (sentiment_analysis): text=the review, language=english/en.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 178 OUT289

Tool Calling - Simple: Knowledge, Places & History (set 5) 10 passed

100.0%

IN73 074 OUT8 160

✓

SimpleBigFiveScoresBFCL/simple_python_229 (get_bigfive_scores): characteristics=[efficient, organized, easy going, compassionate], scale omitted or medium.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 304 OUT708

✓

SimpleChristianityHistoryBFCL/simple_python_249 (religion.history_info): religion=Christianity, till_century=14, include_people omitted or false.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 293 OUT1 176

✓

SimpleEinsteinContributionBFCL/simple_python_247 (historical_contrib.get_contrib): scientist=Albert Einstein, date=1915-03-17 (variants), category omitted or all.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 310 OUT2 523

✓

SimpleHistoricalGdpBFCL/simple_python_237 (get_historical_GDP): country=United States/US, start_year=1960, end_year=2000.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 315 OUT273

✓

SimpleMriReportBFCL/simple_python_218 (patient.get_mri_report): patient_id=546382, mri_type omitted or brain, status=concluded.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 324 OUT494

✓

SimpleNeuronalActivityBFCL/simple_python_220 (calculate_neuronal_activity): input_synaptic_rate=200, weight=0.5, decay_rate=0.1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 334 OUT1 403

✓

SimpleReadingPreferenceBFCL/simple_python_225 (psych_research.get_preference): category=reading, option_one=digital[reading], option_two=physical book(s), demographic omitted or all.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 297 OUT279

✓

SimpleRelativityInventionBFCL/simple_python_248 (science_history.get_invention): invention_name=theory of relativity, want_year=true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 287 OUT341

✓

SimpleRenewableEmissionSavingsBFCL/simple_python_202 (calculate_emission_savings): energy_type=renewable, usage_duration=3, region=California/CA. EXPECTED MODEL MISS (kept, do not loosen): the prompt says "renewable energy sources" generically; models often over-specify a concrete type (e.g. "solar"). Ground truth wants "renewable".1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 315 OUT529

✓

SimpleZodiacCompatibilityBFCL/simple_python_226 (get_zodiac_compatibility): sign1=Aries, sign2=Gemini, scale omitted or percentage.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 295 OUT434

Tool Calling - Simple: Law, Finance & Records (set 1) 8 passed2 failed

80.0%

IN73 006 OUT5 613

✓

SimpleAirQualityByDateBFCL/simple_python_196 (air_quality): location London, date reformatted to 08-16-2022.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 287 OUT562

✓

SimpleCivilCaseDetailsWithDissentBFCL/simple_python_167 (law.civil.get_case_details): case_title Roe v. Wade, include_dissent true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 287 OUT366

×

SimpleCompoundedInterestQuarterlyBFCL/simple_python_153 (calculate_compounded_interest): principal 5000, rate 0.03, time 5, n 4 (quarterly).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.rate == 0.03 Values: args.rate = 3

Failed

IN7 320 OUT1 363

✓

SimpleCrimeRateByCityBFCL/simple_python_164 (get_crime_rate): city San Francisco, state California/CA, type violent/omitted, year 2020.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 297 OUT483

×

SimpleCriminalRecordsByPersonBFCL/simple_python_158 (get_criminal_records): name Mr. X, location New York, NY, from_year 2012, to_year 2015.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.name?.ToLower()).AnyOf(["mr. x", "mr x"])

Failed

IN7 318 OUT563

✓

SimpleDetailedWeatherForecastHoursBFCL/simple_python_185 (detailed_weather_forecast): location New York, duration 72 hours, include_precipitation true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 290 OUT535

✓

SimpleFutureValueByDurationBFCL/simple_python_155 (calculate_future_value): initial_investment 1000, interest_rate 0.05, duration 2, compounded omitted or 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 309 OUT340

✓

SimpleLawsuitDetailsWithExtraFieldsBFCL/simple_python_178 (get_lawsuit_details): case_number LAX2019080202, court_location Los Angeles, additional_details omitted or full default list.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 307 OUT404

✓

SimpleLawsuitSearchByFilingDateBFCL/simple_python_168 (lawsuit_search): company Google/GOOG, start_date 01-01-2021, location California, status ongoing/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 316 OUT444

✓

SimpleLawyerExperienceMonthsBFCL/simple_python_175 (lawyer.get_experience): name John Doe, law_type Bankruptcy.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 275 OUT553

Tool Calling - Simple: Law, Finance & Records (set 2) 10 passed

100.0%

IN70 833 OUT3 513

✓

SimpleAirQualityIndexByTimeBFCL/simple_python_197 (get_air_quality_index): location San Diego, time 12pm/12:00.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 055 OUT192

✓

SimpleCivilCasesRetrieveTheftBFCL/simple_python_165 (civil_cases.retrieve): year 2020, crime_type theft, location Los Angeles.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 070 OUT239

✓

SimpleFindNearbyDivorceLawyerBFCL/simple_python_166 (lawyer.find_nearby): city Chicago IL, specialty [Divorce], fee 400. NOTE: single-call contract kept intentionally; the model occasionally emits a spurious second call (no other visible tool fits this query), which is a model-reliability signal, not a test defect.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 088 OUT646

✓

SimpleGenerateLawContractRentalBFCL/simple_python_162 (generate_law_contract): parties [John, Alice], contract_type Rental Agreement, location California/CA.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 069 OUT356

✓

SimpleLawsuitDetailsFindByCompanyBFCL/simple_python_176 (lawsuit_details.find): company_name Apple Inc., year 2010, case_type Patent/IPR.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 081 OUT402

✓

SimpleLawsuitDetailsWithVerdictBFCL/simple_python_181 (get_lawsuit_details): case_number 123456-ABC, court_location Los Angeles, with_verdict true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 093 OUT313

✓

SimpleLocalNurseryAnnualPlantsBFCL/simple_python_193 (local_nursery.find): location Toronto, plant_types [Annual].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 065 OUT289

✓

SimplePropertyRecordsWithOwnerBFCL/simple_python_163 (property_records.get): address 123 main street, parcel_number 1234567890, county Santa Clara, include_owner true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 141 OUT408

✓

SimpleRoiSavingsAccountBFCL/simple_python_150 (calculate_roi): deposit 1000, annual_interest_rate 0.03, years 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 103 OUT277

✓

SimpleWeatherForecastDetailedDaysBFCL/simple_python_189 (weather_forecast_detailed): location New York, days 3, details true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 068 OUT391

Tool Calling - Simple: Law, Finance & Records (set 3) 10 passed

100.0%

IN71 800 OUT6 536

✓

SimpleCaseInfoVictimBFCL/simple_python_160 (get_case_info): docket 2022/AL2562, court California/CA, info_type victim.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 202 OUT675

✓

SimpleCrimeStatuteTheftBFCL/simple_python_161 (crime_statute_lookup): jurisdiction California/CA, crime theft, detail_level detailed.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 178 OUT1 285

✓

SimpleCurrentWeatherCelsiusBFCL/simple_python_186 (current_weather_condition): city Tokyo, country Japan, measurement c/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 178 OUT475

✓

SimpleEnvironmentalAirQualityDaysBFCL/simple_python_199 (environmental_data.air_quality_index): location San Jose, days 3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 162 OUT349

✓

SimpleFelonyCrimeRecordBFCL/simple_python_156 (crime_record.get_record): case_number CA123456, county San Diego, details true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 194 OUT500

✓

SimpleHighestGrossingBankBFCL/simple_python_151 (highest_grossing_banks): country U.S variants, year 2020, top_n 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 186 OUT1 047

✓

SimpleHistoricalFraudCaseBFCL/simple_python_170 (law_case_search.find_historical): subject fraud, from_year 2010, to_year 2015.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 194 OUT393

✓

SimpleLawsuitCasesByCompanyYearBFCL/simple_python_177 (get_lawsuit_cases): company_name Facebook, year 2018, status all/omitted. EXPECTED MODEL MISS (kept, do not loosen): "Patent" is the case TYPE, not the status (enum is open/closed/all); status should be omitted/"all". Models often wrongly set status="Patent".1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 177 OUT801

✓

SimpleLawsuitInfoByCaseNumberBFCL/simple_python_182 (lawsuit_info): case_number XYZ123, year omitted/2023, location omitted/all.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 157 OUT287

✓

SimplePlantsForSlopeErosionBFCL/simple_python_194 (get_plants_for_slope): slope_type hill/steep/moderate, num_results 3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 172 OUT724

Tool Calling - Simple: Law, Finance & Records (set 4) 10 passed

100.0%

IN73 536 OUT5 552

✓

SimpleActDetailsAmendmentBFCL/simple_python_159 (get_act_details): act_name Criminal Law Amendment Act, amendment_year 2013.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 328 OUT539

✓

SimpleCheckFeloniesByNameBFCL/simple_python_157 (criminal_history.check_felonies): full_name John Doe, birth_date 01-01-1980, state California/CA.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 368 OUT400

✓

SimpleCourtCaseSearchNoFullTextBFCL/simple_python_169 (court_case.search): docket_number 123456, location Texas, full_text false/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 355 OUT361

✓

SimpleCurrentWeatherTempHumidityBFCL/simple_python_187 (get_current_weather): location Seattle, include_temperature true/omitted, include_humidity true/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 303 OUT276

✓

SimpleLawsuitCheckClosedStatusBFCL/simple_python_184 (lawsuit.check_case): case_id 1234, closed_status true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 341 OUT298

✓

SimpleLawsuitsSearchByCompanyLocationBFCL/simple_python_180 (lawsuits_search): company_name Google, location California/CA, year 2020, case_type all/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 336 OUT265

✓

SimpleMutualFundBalanceBFCL/simple_python_152 (calculate_mutual_fund_balance): investment_amount 50000, annual_yield 0.05, years 3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 366 OUT335

✓

SimpleSlopeGradientCoordinatesBFCL/simple_python_192 (calculate_slope_gradient): point1 [40.7128,-74.006], point2 [34.0522,-118.2437], unit degree/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 431 OUT566

✓

SimpleStateLawCaseYearRangeBFCL/simple_python_173 (law_case_search): topic land disputes, year_range [2015,2021], location New York, judicial_system state.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 377 OUT584

✓

SimpleTopConstitutionalCasesBFCL/simple_python_174 (get_top_cases): field_of_law constitutional law, top_number 10, country China/CN.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 331 OUT1 928

Tool Calling - Simple: Law, Finance & Records (set 5) 8 passed2 failed

80.0%

IN79 647 OUT6 042

✓

SimpleCarbonFootprintLifestyleBFCL/simple_python_195 (calculate_carbon_footprint): daily_miles 20, meat_meals_per_week 3, annual_trash_weight 500, flights_per_year omitted/0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 263 OUT1 228

✓

SimpleDailyWaterIntakeWeightBFCL/simple_python_198 (calculate_daily_water_intake): weight 70, activity_level omitted/moderate, climate omitted/temperate.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 189 OUT650

✓

SimpleFetchLawCaseByNumberBFCL/simple_python_171 (fetch_law_case_details): case_number 43403, court New York, year 2018.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 233 OUT281

✓

SimpleFutureValueByPresentValueBFCL/simple_python_154 (calculate_future_value): present_value 5000, annual_interest_rate 0.05, years 10, compounds_per_year omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 242 OUT732

✓

SimpleHumidityForecastDaysBFCL/simple_python_188 (weather.humidity_forecast): location Miami, days 7, min_humidity omitted/0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 198 OUT279

×

SimpleLatestCourtCaseCompaniesBFCL/simple_python_179 (find_latest_court_case): company1 Apple, company2 Samsung, country USA/omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 974 OUT581

×

SimpleLawsuitSearchByCountyBFCL/simple_python_183 (lawsuit_search): entity Apple, county Santa Clara County, state California/omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 949 OUT490

✓

SimpleLegalCaseFetchDetailsBFCL/simple_python_172 (legal_case.fetch): case_id R vs Adams, details true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 198 OUT549

✓

SimpleParkElevationAreaBFCL/simple_python_190 (park_information): park_name Yellowstone, information [Elevation, Area] (order-independent).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 192 OUT781

✓

SimpleTallestMountainsRadiusBFCL/simple_python_191 (locate_tallest_mountains): location Denver, radius 50, amount 5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 209 OUT471

Tool Calling - Simple: Live Answer Tool (isolated) (set 1) 1 passed

100.0%

IN4 008 OUT318

✓

SimpleAnswerStringLogistic1BFCL/live_simple_32-9-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L113: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 008 OUT318

Tool Calling - Simple: Live Answer Tool (isolated) (set 2) 1 passed

100.0%

IN3 975 OUT372

✓

SimpleAnswerStringLogistic2BFCL/live_simple_33-10-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L114: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 975 OUT372

Tool Calling - Simple: Live Answer Tool (isolated) (set 3) 1 skipped

-

IN0 OUT0

–

SimpleAnswerStringValidBFCL/live_simple_34-11-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L115: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Harness-semantic mismatch (item flaw for our runner): the query ('...Valid?') is directly answerable, and our system prompt permits answering in plain text when no tool is needed, so the model reasonably returns 0 tool calls instead of routing to the answer_string meta-tool. Not a deterministic tool-routing test. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Simple: Live Answer Tool (isolated) (set 4) 1 passed

100.0%

IN4 037 OUT391

✓

SimpleAnswerStringWhatIsBFCL/live_simple_35-12-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L116: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 037 OUT391

Tool Calling - Simple: Live Assistants, Weather & Devices (set 1) 2 passed1 failed

66.7%

IN14 737 OUT704

✓

SimpleChangeFoodBurgerBFCL/live_simple_20-4-0 (change_food): food_item='burger', modification_request='no onions, extra cheese'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 899 OUT186

×

SimpleThinqAirCleanHighBFCL/live_simple_40-17-0 (ThinQ_Connect): AIR_CLEAN mode, HIGH wind, monitoring on, air clean POWER_ON. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. windStrength broadened (high|strongest|strong) for "strongest wind strength". EXPECTED MODEL MISS: 27B sets airConJobMode=AIR_CLEAN but omits airCleanOperationMode=POWER_ON — the prompt asks to "start the air purifying function", so POWER_ON is the fair GT; kept strict (a real tool-use miss).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.airCleanOperationMode?.ToLower()).AnyOf(["power_on"])

Failed

IN4 939 OUT249

✓

SimpleWeatherTelAvivBFCL/live_simple_4-3-0 (get_current_weather): location='Tel Aviv, Israel', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 899 OUT269

Tool Calling - Simple: Live Assistants, Weather & Devices (set 10) 3 passed

100.0%

IN13 952 OUT1 475

✓

SimpleChafodOrderPizzaBFCL/live_simple_23-5-1 (ChaFod): TheFod='PIZZA'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 626 OUT940

✓

SimpleUberEatMcdonaldsPizzaBFCL/live_simple_29-7-2 (uber.eat.order): restaurant="McDonald's", items=[pizza], quantities=[1].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 660 OUT261

✓

SimpleWeatherShanghaiBFCL/live_simple_13-3-9 (get_current_weather): location='Shanghai, China', unit=celsius. Prompt is Chinese, copied verbatim.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 666 OUT274

Tool Calling - Simple: Live Assistants, Weather & Devices (set 11) 3 passed

100.0%

IN15 100 OUT1 373

✓

SimpleChafodSwitchBurgerBFCL/live_simple_24-5-2 (ChaFod): TheFod='BURGER'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 005 OUT788

✓

SimpleFindBeerSierraNevadaBFCL/live_simple_48-21-0 (find_beer): brewery='Sierra Nevada', taste=bitter, aroma=hoppy, color=pale, style=lager; defaults omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 065 OUT301

✓

SimpleWeatherChennaiBFCL/live_simple_15-3-11 (get_current_weather): location='Chennai, Tamil Nadu', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 030 OUT284

Tool Calling - Simple: Live Assistants, Weather & Devices (set 12) 2 passed1 failed

66.7%

IN15 213 OUT1 497

✓

SimpleChafodGreensSaladBFCL/live_simple_25-5-3 (ChaFod): TheFod='SALAD' (greens mapped to salad).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 030 OUT150

×

SimpleFindBeerCreekPorterBFCL/live_simple_49-21-1 (find_beer): brewery='Creek', taste='slightly bitter', aroma=chocolatey, color=dark, style=porter; defaults omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.taste?.ToLower()).AnyOf(["slightly bitter"])

Failed

IN5 120 OUT945

✓

SimpleWeatherQuintanaRooBFCL/live_simple_19-3-15 (get_current_weather): location='Quintana Roo, Mexico', unit=celsius.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 063 OUT402

Tool Calling - Simple: Live Assistants, Weather & Devices (set 13) 1 passed1 failed

50.0%

IN9 605 OUT725

✓

SimpleLexListExportsV2BFCL/live_simple_30-8-0 (aws.lexv2_models.list_exports): botId='my-bot-id', botVersion='v2', sortBy=ASC or omitted, others omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 838 OUT281

×

SimpleWeatherLangSonBFCL/live_simple_16-3-12 (get_current_weather): location='Lang Son, Vietnam', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.location?.ToLower()).AnyOf(["lang son, vietnam"])

Failed

IN4 767 OUT444

Tool Calling - Simple: Live Assistants, Weather & Devices (set 14) 2 passed

100.0%

IN9 600 OUT669

✓

SimpleLexListExportsDescBFCL/live_simple_31-8-1 (aws.lexv2_models.list_exports): botId='B12345', botVersion='v1', sortBy=DESC, others omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 842 OUT444

✓

SimpleWeatherBostonBFCL/live_simple_17-3-13 (get_current_weather): location='Boston, MA', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 758 OUT225

Tool Calling - Simple: Live Assistants, Weather & Devices (set 15) 2 passed

100.0%

IN8 714 OUT478

✓

SimpleGetUserInfoBlackBFCL/live_simple_0-0-0 (get_user_info): user_id=7890, special='black'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 365 OUT182

✓

SimpleWeatherMoscowBFCL/live_simple_18-3-14 (get_current_weather): location='Moscow, Russia', unit=celsius.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 349 OUT296

Tool Calling - Simple: Live Assistants, Weather & Devices (set 16) 3 passed

100.0%

IN14 597 OUT1 700

✓

SimpleFetchWeatherCoordsBFCL/live_simple_39-16-0 (fetch_weather_data): latitude=37.8651, longitude=-119.5383; url/units/language defaults omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 879 OUT670

✓

SimpleGithubStarGorillaBFCL/live_simple_1-1-0 (github_star): repos='ShishirPatil/gorilla,gorilla-llm/gorilla-cli', aligned=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 881 OUT358

✓

SimpleWeatherBangkokBFCL/live_simple_14-3-10 (get_current_weather): location='Bangkok, Thailand', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 837 OUT672

Tool Calling - Simple: Live Assistants, Weather & Devices (set 17) 3 passed

100.0%

IN13 882 OUT1 094

✓

SimpleCarbonIntensityGBBFCL/live_simple_50-22-0 (get_latest_carbon_intensity): zone='Great Britain' or 'GB'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 590 OUT235

✓

SimpleMultiplyThreeTwoBFCL/live_simple_47-20-0 (multiply): a=3, b=2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 592 OUT163

✓

SimpleWeatherOpenMeteoMariposaBFCL/live_simple_38-15-0 (get_current_weather, open-meteo variant): location='Mariposa, CA', unit=fahrenheit; url default omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 700 OUT696

Tool Calling - Simple: Live Assistants, Weather & Devices (set 2) 3 passed

100.0%

IN14 696 OUT832

✓

SimpleChangeFoodChickenSpicyBFCL/live_simple_21-4-1 (change_food): food_item='chicken dish', modification_request='extra spicy'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 921 OUT193

✓

SimpleThinqAirDryMidBFCL/live_simple_41-17-1 (ThinQ_Connect): AIR_DRY mode, MID wind (or omitted), all else optional. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. windStrength broadened (mid|medium) for "medium wind strength"; air-clean off-state accepts off/power_off.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 889 OUT215

✓

SimpleWeatherDivinopolisBFCL/live_simple_5-3-1 (get_current_weather): location='Divinopolis, MG' (verbatim), unit=fahrenheit or omitted. Prompt is Portuguese, copied verbatim.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 886 OUT424

Tool Calling - Simple: Live Assistants, Weather & Devices (set 3) 3 passed

100.0%

IN14 717 OUT1 784

✓

SimpleParseAnswerNoDefineBFCL/live_simple_36-13-0 (parseAnswer): answer='The text does not define logistic regression, therefore I cannot answer this question.'1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 914 OUT1 136

✓

SimpleThinqKoreanTarget22BFCL/live_simple_42-17-2 (ThinQ_Connect): coolTargetTemperature=22 required; mode COOL and wind MID optional. Prompt is Korean, copied verbatim. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. Prompt is dehumidify (제습) so GT's optional airConJobMode=COOL is a source oddity — accept cool/dehumidify/air_dry (optional). windStrength MID broadened (mid|medium). The "22도" target-temp is accepted in either coolTargetTemperature or targetTemperature (the two are genuinely ambiguous for "target temperature").1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 920 OUT365

✓

SimpleWeatherSanFranciscoBFCL/live_simple_6-3-2 (get_current_weather): location='San Francisco, CA', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 883 OUT283

Tool Calling - Simple: Live Assistants, Weather & Devices (set 4) 3 passed

100.0%

IN14 579 OUT1 593

✓

SimpleParseAnswerNoDefineValidityBFCL/live_simple_37-14-0 (parseAnswer): answer='The text does not define logistic regression, therefore I cannot answer this question.'1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 873 OUT961

✓

SimpleThinqCoolHighWindBFCL/live_simple_43-17-3 (ThinQ_Connect): HIGH wind required; COOL mode and coolTargetTemperature=24 optional. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. air-clean off-state broadened (power_off|off) — the model files the unrequested clean-mode as "off".1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 868 OUT192

✓

SimpleWeatherRigaBFCL/live_simple_7-3-3 (get_current_weather): location='Riga, Latvia'; unit fahrenheit/celsius/omitted all accepted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 838 OUT440

Tool Calling - Simple: Live Assistants, Weather & Devices (set 5) 3 passed

100.0%

IN15 192 OUT764

✓

SimpleThinqTurnedOnCoolBFCL/live_simple_44-18-0 (ThinQ_Connect): airConOperationMode=POWER_ON required; COOL currentJobMode optional. NOTE: tool flattened to nullable top-level params (see struct comment) so a valid partial call is one tool call, not a forced retry. windStrength/airClean off-state broadened to synonyms (mid|medium, stop|off|power_off).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 041 OUT210

✓

SimpleUberComfortAddisonBFCL/live_simple_2-2-0 (uber.ride): loc='2020 Addison Street, Berkeley, CA, USA', type=comfort, time=600.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 103 OUT241

✓

SimpleWeatherLondonBFCL/live_simple_8-3-4 (get_current_weather): location='London, UK' or 'London, England'; unit fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 048 OUT313

Tool Calling - Simple: Live Assistants, Weather & Devices (set 6) 2 passed1 failed

66.7%

IN15 227 OUT898

×

SimpleThinqCoolAirCleanStartBFCL/live_simple_45-18-1 (ThinQ_Connect): airCleanOperationMode=START, airConOperationMode=POWER_ON, coolTargetTemperature=20. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. airCleanOperationMode "air cleaning on" broadened (start|on|power_on). airConOperationMode=POWER_ON and coolTargetTemperature=20 kept strict — both fair GT. EXPECTED MODEL MISS: 27B gets job-mode + air-clean START + temp, but omits airConOperationMode=POWER_ON.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.airConOperationMode?.ToLower()).AnyOf(["power_on"])

Failed

IN5 093 OUT310

✓

SimpleUberPlusBakerBFCL/live_simple_3-2-1 (uber.ride): loc='221B Baker Street, Berkeley, CA, USA', type=plus, time=600.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 105 OUT290

✓

SimpleWeatherHyderabadBFCL/live_simple_9-3-5 (get_current_weather): location='Hyderabad, India', unit=fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 029 OUT298

Tool Calling - Simple: Live Assistants, Weather & Devices (set 7) 2 passed1 failed

66.7%

IN15 249 OUT1 711

✓

SimpleThinqFlatCoolCleanBFCL/live_simple_46-19-0 (ThinQ_Connect, flat params): COOL, MID, monitoring on, clean START, POWER_ON, power-save off, target 24C.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 144 OUT594

✓

SimpleUberPlusVietnameseBFCL/live_simple_26-6-0 (uber.ride): loc Vietnamese/English form, type=plus, time=10 (minutes). Prompt is Vietnamese, copied verbatim.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 067 OUT874

×

SimpleWeatherYosemiteMariposaBFCL/live_simple_10-3-6 (get_current_weather): location='Mariposa, CA', unit=celsius.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.location?.ToLower()).AnyOf(["mariposa, ca"])

Failed

IN5 038 OUT243

Tool Calling - Simple: Live Assistants, Weather & Devices (set 8) 2 passed1 failed

66.7%

IN15 066 OUT1 148

×

SimpleThinqCoolScheduledBFCL/live_simple_51-23-0 (ThinQ_Connect): COOL, HIGH wind, POWER_ON, power-save on, target 22C, start in 1h30m. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. airConOperationMode=POWER_ON kept strict ("Start in 1h30m" schedules a power-on). EXPECTED MODEL MISS: 27B gets job-mode COOL + wind + power-save + temp + schedule, but omits airConOperationMode=POWER_ON.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.airConOperationMode?.ToLower()).AnyOf(["power_on"])

Failed

IN5 096 OUT502

✓

SimpleUberEatPitadaBFCL/live_simple_27-7-0 (uber.eat.order): restaurant='uber pitada', items=[burgers, chicken wings], quantities=[5, 6].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 005 OUT397

✓

SimpleWeatherNaplesBFCL/live_simple_11-3-7 (get_current_weather): location='Naples, FL' or 'Naples, Florida'; unit fahrenheit or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 965 OUT249

Tool Calling - Simple: Live Assistants, Weather & Devices (set 9) 3 passed

100.0%

IN14 050 OUT1 312

✓

SimpleChafodWhopperBurgerBFCL/live_simple_22-5-0 (ChaFod): TheFod='BURGER' (Whopper mapped to burger).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 639 OUT191

✓

SimpleUberEatKfcChineseBFCL/live_simple_28-7-1 (uber.eat.order): KFC order, 4 items, quantities [10,50,30,90]. Prompt is Chinese, copied verbatim.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 713 OUT386

✓

SimpleWeatherNewYorkBFCL/live_simple_12-3-8 (get_current_weather): location='New York, NY', unit=celsius.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 698 OUT735

Tool Calling - Simple: Live Media, Weather & Data (set 2) 10 passed

100.0%

IN76 081 OUT3 584

✓

SimpleCreditQuoteAutoCreditoBFCL/live_simple_67-31-0 (obtener_cotizacion_de_creditos): monto=1000000, plazo=12, producto='auto', año=2024, enganche=0.2.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 696 OUT1 187

✓

SimpleCurrentLocDmsBFCL/live_simple_87-48-0 (get_current_loc): coordinate_format='DMS', include_altitude=true, timeout default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 583 OUT308

✓

SimpleGetMoviesMumbaiTodayBFCL/live_simple_59-28-0 (get_movies): city='Mumbai', optionals default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 551 OUT254

✓

SimplePlayArtistKaarijaTracksBFCL/live_simple_93-54-0 (play_artist): artist_name='Käärijä'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 566 OUT137

✓

SimpleRetrieveAnalysisSuperfancyBFCL/live_simple_69-33-0 (analysis_api.AnalysisApi.retrieve_analysis): project/component/vulnerability UUIDs.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 681 OUT390

✓

SimpleSensorAlertsPerPage10BFCL/live_simple_70-34-0 (get_sensor_alerts): perPage=10, sensorSerial='Q3CC-CRT3-SZ2G', rest omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 629 OUT401

✓

SimpleSitefinityNewsItemJulian12BFCL/live_simple_81-42-0 (sitefinity_create_contentitem): ContentItem='NewsItem', Title='Julian is testing12', rest omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 632 OUT261

✓

SimpleTodoAddShopping9pmB25BFCL/live_simple_55-25-1 (todo): type='add', content='go for shopping at 9 pm'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 575 OUT154

✓

SimpleTodoDeleteTodoRandomBFCL/live_simple_54-25-0 (todo): type='delete', content='todo random'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 578 OUT140

✓

SimpleWeatherGetLondonUkBFCL/live_simple_77-38-0 (weather.get): city='London', country='GB' (from UK), units/forecast default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 590 OUT352

Tool Calling - Simple: Live Media, Weather & Data (set 3) 9 passed1 failed

90.0%

IN71 631 OUT2 903

×

SimpleExtractMillennialsAppleBFCL/live_simple_71-35-0 (extract_parameters_v1): demographics=[millennials], targets=[brand:Apple], metrics=[view], min_date=2022-07-01. NOTE: targets @param clarified to '<attribute>:<entity>' order (was ambiguous 'target:target_type', which made the model emit the reversed 'Apple:brand'); GT 'brand:Apple' kept strict.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.metrics[0]?.ToLower()).AnyOf(["view"])

Failed

IN7 194 OUT852

✓

SimpleFetchPersonId4BFCL/live_simple_91-52-0 (fetchPerson): id=4.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 137 OUT299

✓

SimpleLogFoodChaiTeaBFCL/live_simple_88-49-0 (log_food): food_name='chai tea', portion_amount=16.0, portion_unit='ounces', meal_name='snack'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 205 OUT201

✓

SimplePlaySongChaChaChaBFCL/live_simple_94-55-0 (play_song): artist_name='Käärijä', song_name='Cha Cha Cha'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 155 OUT138

✓

SimpleSendEmailAndyForecastBFCL/live_simple_78-39-0 (send_email): to='andy@gorilla.ai', subject='Sales Forecast Request', body set, cc/bcc/attachments omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 203 OUT280

✓

SimpleSitefinityNewsJulianTestingBFCL/live_simple_82-43-0 (sitefinity_create_contentitem): ContentItem='News', Title='Julian is testing', rest omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 171 OUT296

✓

SimpleTemperatureParisFranceBFCL/live_simple_90-51-0 (temperature): location='Paris, France', units/time default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 129 OUT287

✓

SimpleTodoManagerAddSleepBFCL/live_simple_57-26-1 (todo_manager.handle_action): type='add', content='go to sleep at 9 pm'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 142 OUT144

✓

SimpleTodoManagerDeleteGymBFCL/live_simple_56-26-0 (todo_manager.handle_action): type='delete', content='go to gym'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 132 OUT132

✓

SimpleTranslateEnglishFrenchBFCL/live_simple_76-37-0 (language_translator.translate): source='en', target='fr', text='What is your name?'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 163 OUT274

Tool Calling - Simple: Live Media, Weather & Data (set 4) 10 passed

100.0%

IN53 288 OUT2 128

✓

SimpleOpenMeteoBerkeleyBFCL/live_simple_85-46-0 (open_meteo_api.fetch_weather_data): coordinates=[37.8715,-122.273], units default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 336 OUT342

✓

SimpleSearchHotelsSanDiegoBFCL/live_simple_79-40-0 (search_hotels): location='San Diego, CA'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 384 OUT208

✓

SimpleSitefinityNewsSleepProductivityBFCL/live_simple_83-44-0 (sitefinity_create_contentitem): full news article with all metadata.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 478 OUT401

✓

SimpleSortArrayAscendingBFCL/live_simple_86-47-0 (sort_array): array=[1,54,3,1,76,2343,21], reverse default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 354 OUT278

✓

SimpleTodoAddGoToGoaBFCL/live_simple_64-29-4 (todo): type='add', content='go to Goa'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 281 OUT129

✓

SimpleTodoAddGymTomorrowBFCL/live_simple_61-29-1 (todo): type='add', content='go to gym tomorrow.'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 280 OUT120

✓

SimpleTodoAddHiCharlieBFCL/live_simple_65-29-5 (todo): type='add', content='Hi charlie'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 291 OUT143

✓

SimpleTodoAddSleep9pmBFCL/live_simple_63-29-3 (todo): type='add', content='go to sleep at 9 pm'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 287 OUT157

✓

SimpleTodoCompleteFinancialReportBFCL/live_simple_60-29-0 (todo): type='complete', content='Submit monthly financial report'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 304 OUT217

✓

SimpleTodoDeleteRaviBFCL/live_simple_62-29-2 (todo): type='delete', content='ravi'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 293 OUT133

Tool Calling - Simple: Live Movies, Music & Booking (set 1) 3 passed

100.0%

IN14 597 OUT696

✓

SimpleMoviesTarantinoWhitakerBFCL/live_simple_208-117-0 (Movies_3_FindMovies): directed_by='Quentin Tarantino', cast='Duane Whitaker', genre dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 886 OUT409

✓

SimplePlaySongRoarBFCL/live_simple_230-121-0 (play_song): song_id='wjeiruhuq_roar', artist='Katy Perry', quality='high', device_id omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 883 OUT191

✓

SimpleSetVolume20BFCL/live_simple_242-126-0 (set_volume): volume=20.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 828 OUT96

Tool Calling - Simple: Live Movies, Music & Booking (set 10) 3 passed

100.0%

IN15 598 OUT901

✓

SimpleHandoverPackageArrivalBFCL/live_simple_257-137-1 (answer_question): statement=package-arrival-time question, urgency/language default. System instructs HANDOVER on question. NOTE: no collider loop observed (single clean call, sibling tests pass); statement is free text -> substring-intent of the key phrase (model drops "Can you tell me").1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 217 OUT331

✓

SimpleMoviesNiteshDirectorBFCL/live_simple_217-117-9 (Movies_3_FindMovies): directed_by='Nitesh Tiwari', genre/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 168 OUT315

✓

SimpleTtsChineseMaleBFCL/live_simple_227-118-1 (text_to_speech.convert): text=Chinese phrase, language='zh-CN', gender='male', format='wav', speed default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 213 OUT255

Tool Calling - Simple: Live Movies, Music & Booking (set 11) 3 passed

100.0%

IN14 947 OUT1 168

✓

SimpleMoviesColaizzoWatkinsBFCL/live_simple_218-117-10 (Movies_3_FindMovies): directed_by='Paul Downs Colaizzo', cast='Michaela Watkins', genre dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 036 OUT641

✓

SimpleRequestsGetBuProductsBFCL/live_simple_229-120-0 (requests.get): anchor='user'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 954 OUT274

✓

SimpleTtsPrettyGirlDefaultBFCL/live_simple_228-119-0 (text_to_speech.convert): text='I am a pretty girl', language/gender/speed default.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 957 OUT253

Tool Calling - Simple: Live Movies, Music & Booking (set 12) 3 passed

100.0%

IN14 662 OUT1 095

✓

SimpleBookFlightParisBFCL/live_simple_233-123-0 (book_flight): departure_location='Paris, France', departure_time='12/03/2023 15:00', return_time omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 923 OUT271

✓

SimpleGetVersionBFCL/live_simple_247-129-0 (version_api.VersionApi.get_version): no arguments.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 835 OUT339

✓

SimpleMoviesSimonCurtisDramaBFCL/live_simple_219-117-11 (Movies_3_FindMovies): directed_by='Simon Curtis', genre='Drama', cast='McKinley Belcher III'.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 904 OUT485

Tool Calling - Simple: Live Movies, Music & Booking (set 13) 3 passed

100.0%

IN15 218 OUT1 946

✓

SimpleAclRetrieveProjectsBFCL/live_simple_248-130-0 (acl_api.AclApi.retrieve_projects): uuid='12', excludeInactive=true, onlyRoot=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 054 OUT635

✓

SimpleBookFlightNewYorkToParisBFCL/live_simple_234-123-1 (book_flight): departure_location='New York, NY', departure_time='20/06/2022 17:00', return_time omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 123 OUT815

✓

SimpleMoviesMackendrickThrillerBFCL/live_simple_220-117-12 (Movies_3_FindMovies): directed_by='Alexander Mackendrick', genre='Thriller', cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 041 OUT496

Tool Calling - Simple: Live Movies, Music & Booking (set 14) 3 passed

100.0%

IN14 821 OUT1 288

✓

SimpleBookFlightJfkBFCL/live_simple_245-127-0 (book_flight): departure_location='JFK', departure_date='16/03/2024', departure_time='14:00'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 995 OUT296

✓

SimpleChatWithBunnyBFCL/live_simple_249-131-0 (chat_with_friend): user_message='Hey bunny, How are you doing'. NOTE: no collider loop observed (single clean call, sibling tests pass); user_message is a free-text passthrough -> substring-intent (model may drop the "Hey bunny," greeting).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 910 OUT345

✓

SimpleMoviesJamesGrayBFCL/live_simple_221-117-13 (Movies_3_FindMovies): directed_by='James Gray', genre/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 916 OUT647

Tool Calling - Simple: Live Movies, Music & Booking (set 15) 2 passed

100.0%

IN9 300 OUT639

✓

SimpleBookFlightNewYorkMarchBFCL/live_simple_246-128-0 (book_flight): departure_location='New York, NY', departure_time='15/03/2024'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 642 OUT297

✓

SimpleMoviesVanOrmanBrownBFCL/live_simple_222-117-14 (Movies_3_FindMovies): directed_by='Thurop Van Orman', cast='Sterling K. Brown', genre dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 658 OUT342

Tool Calling - Simple: Live Movies, Music & Booking (set 16) 1 passed1 skipped

100.0%

IN4 618 OUT566

✓

SimpleMoviesImaginativeFantasyBFCL/live_simple_223-117-15 (Movies_3_FindMovies): genre='Fantasy' ('Imaginative fiction'), director/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 618 OUT566

–

SimpleRescheduleGymSessionBFCL/live_simple_231-122-0 (reschedule_event): event_identifier='456123', new_datetime='2022-10-30T16:30:00Z'.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: source flaw: relative-date arithmetic is inconsistent — prompt states 'Monday 2022-10-20' (that date is actually a Thursday) and the GT 'next Thursday' resolves to 2022-10-30 (a Sunday), so the exact new_datetime cannot be derived deterministically.

Skipped

IN0 OUT0

Tool Calling - Simple: Live Movies, Music & Booking (set 17) 2 passed

100.0%

IN9 266 OUT737

✓

SimpleMoviesTimBurtonDeppBFCL/live_simple_224-117-16 (Movies_3_FindMovies): directed_by='Tim Burton', genre='Offbeat', cast='Johnny Depp'.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 589 OUT513

✓

SimpleRescheduleMedAppointmentBFCL/live_simple_232-122-1 (reschedule_event): event_identifier='med123', new_datetime='2021-04-15T13:45:00Z'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 677 OUT224

Tool Calling - Simple: Live Movies, Music & Booking (set 18) 2 passed

100.0%

IN8 954 OUT441

✓

SimpleMoviesKirillComedyDramaBFCL/live_simple_225-117-17 (Movies_3_FindMovies): directed_by='Kirill Mikhanovsky', genre='Comedy-drama', cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 512 OUT328

✓

SimpleSetVolume70BFCL/live_simple_244-126-2 (set_volume): volume=70.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 442 OUT113

Tool Calling - Simple: Live Movies, Music & Booking (set 2) 3 passed

100.0%

IN14 642 OUT648

✓

SimpleMoviesLeitchTuisanoBFCL/live_simple_209-117-1 (Movies_3_FindMovies): directed_by='David Leitch', cast='Lori Pelenise Tuisano', genre dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 911 OUT354

✓

SimpleSetVolume30BFCL/live_simple_243-126-1 (set_volume): volume=30.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 864 OUT147

✓

SimpleSpotifyFriendsMarshmelloBFCL/live_simple_235-124-0 (play_spotify_song): query='track:Friends artist:Marshmello', shuffle/volume default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 867 OUT147

Tool Calling - Simple: Live Movies, Music & Booking (set 3) 3 passed

100.0%

IN15 207 OUT674

✓

SimpleFlightStatusIndigo6E123BFCL/live_simple_251-133-0 (flight.status.check): flight_id='6E123', passenger_name='Karan', ticket_number='IND4567'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 102 OUT285

✓

SimpleMoviesDramaBFCL/live_simple_210-117-2 (Movies_3_FindMovies): genre='Drama', director/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 052 OUT208

✓

SimpleSpotifyDilNu124BFCL/live_simple_236-124-1 (play_spotify_song): query='track:Dil Nu artist:Maninder Buttar', shuffle/volume default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 053 OUT181

Tool Calling - Simple: Live Movies, Music & Booking (set 4) 3 passed

100.0%

IN15 203 OUT1 348

✓

SimpleFlightStatusIndigoAirlineBFCL/live_simple_252-134-0 (flight.status.check): airlines='Indigo', person_name='Karan', ticket_number='IND4567'. NOTE: tool doc says "if only one name mentioned ... use LNU", so "Karan LNU" is equivalent-correct per the documented convention -> accept both.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 096 OUT314

✓

SimpleMoviesComedyCordenBFCL/live_simple_211-117-3 (Movies_3_FindMovies): genre='Comedy' (prompt 'Comic'), cast='James Corden', director dontcare. NOTE: args are correct; single-call contract stays strict. The 27B occasionally emits a 2nd call (over-call) -> legit intermittent fail (kept as signal).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 072 OUT899

✓

SimpleSpotifyWreckingBallBFCL/live_simple_237-125-0 (play_spotify_song): query='track:Wrecking Ball artist:Miley Cyrus', device_id/play default. NOTE: GT accepts only the structured 'track:.. artist:..' form (kept strict). The 27B occasionally drops the format (e.g. "wrecking ball Miley Cyrus") -> legit intermittent miss (kept as signal).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 035 OUT135

Tool Calling - Simple: Live Movies, Music & Booking (set 5) 3 passed

100.0%

IN15 161 OUT775

✓

SimpleMoviesComedyEdgarWrightBFCL/live_simple_212-117-4 (Movies_3_FindMovies): directed_by='Edgar Wright', genre='Comedy', cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 037 OUT352

✓

SimpleSpotifyDilNu125BFCL/live_simple_238-125-1 (play_spotify_song): query='track:Dil Nu artist:Maninder Buttar', device_id/play default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 043 OUT190

✓

SimpleUberEatsBigMacBFCL/live_simple_250-132-0 (uber.eat.order): restaurants='McDonald's', items=[Big Mac, McFlurry], quantities=[2,1].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 081 OUT233

Tool Calling - Simple: Live Movies, Music & Booking (set 6) 3 passed

100.0%

IN15 160 OUT889

✓

SimpleMoviesOffbeatTimBurtonBFCL/live_simple_213-117-5 (Movies_3_FindMovies): directed_by='Tim Burton', genre='Offbeat', cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 056 OUT453

✓

SimpleSpotifyShapeOfYouBFCL/live_simple_239-125-2 (play_spotify_song): query='track:Shape of You artist:Ed Sheeran' or 'Shape of You'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 036 OUT199

✓

SimpleUberEatsBurgersWingsBFCL/live_simple_253-135-0 (uber.eat.order): restaurants='McDonald's', items=[burgers, chicken wings], quantities=[5,6].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 068 OUT237

Tool Calling - Simple: Live Movies, Music & Booking (set 7) 3 passed

100.0%

IN14 960 OUT1 576

✓

SimpleMoviesComicNiteshTiwariBFCL/live_simple_214-117-6 (Movies_3_FindMovies): directed_by='Nitesh Tiwari', genre='Comedy' or 'Comedy-drama', cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 039 OUT1 018

✓

SimpleSpotifyBabySharkBFCL/live_simple_240-125-3 (play_spotify_song): query='Baby Shark' (artist unknown).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 949 OUT298

✓

SimpleUserMandatesPendingBFCL/live_simple_254-136-0 (user.mandates): user_id='U123', status='pending'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 972 OUT260

Tool Calling - Simple: Live Movies, Music & Booking (set 8) 3 passed

100.0%

IN14 898 OUT879

✓

SimpleMoviesFantasyTonightBFCL/live_simple_215-117-7 (Movies_3_FindMovies): genre='Fantasy', director/cast dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 968 OUT481

✓

SimpleSpotifyJohnnyPapaBFCL/live_simple_241-125-4 (play_spotify_song): query='Johnny Johnny Yes Papa' (artist unknown).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 945 OUT189

✓

SimpleUserMandatesActiveBFCL/live_simple_255-136-1 (user.mandates): user_id='U123456', status='active' or omitted (default).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 985 OUT209

Tool Calling - Simple: Live Movies, Music & Booking (set 9) 2 passed1 failed

66.7%

IN15 576 OUT946

×

SimpleHandoverComplaintStatusBFCL/live_simple_256-137-0 (answer_question): statement=user complaint-status question, urgency/language default. System instructs HANDOVER on question.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.statement?.ToLower()).AnyOf(["i have had submit a compaint and i like to check its status"])

Failed

IN5 217 OUT370

✓

SimpleMoviesActionLeitchKingBFCL/live_simple_216-117-8 (Movies_3_FindMovies): directed_by='David Leitch', genre='Action', cast='Alex King'.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 176 OUT266

✓

SimpleTtsPrettyGirlSpanishBFCL/live_simple_226-118-0 (text_to_speech.convert): text='I am a pretty girl' or Spanish translation, language='es-ES', gender/format/speed default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 183 OUT310

Tool Calling - Simple: Live Services, Weather & Commands (set 1) 4 passed

100.0%

IN18 934 OUT1 202

✓

SimpleExtractChesterJaneBFCL/live_simple_189-114-0 (extractor.extract_information): [Chester,42],[Jane,43] (Jane a year older).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 780 OUT312

✓

SimpleServiceIdIroningUnitIntBFCL/live_simple_174-100-0 (get_service_id): ironing service -> service_id=2, unit omitted/1. Thai prompt.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 727 OUT301

✓

SimpleWeatherNewYorkBFCL/live_simple_193-116-1 (Weather_1_GetWeather): city=New York, date=2023-03-08.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 713 OUT286

✓

SimpleWeatherPacificaBFCL/live_simple_192-116-0 (Weather_1_GetWeather): city=Pacifica, date=2023-04-11.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 714 OUT303

Tool Calling - Simple: Live Services, Weather & Commands (set 10) 7 passed2 failed

77.8%

IN41 476 OUT3 993

✓

SimpleCmdDockerPsBFCL/live_simple_172-99-6 (cmd_controller.execute): is Docker running -> 'docker ps'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 573 OUT233

✓

SimpleCmdDockerStartBFCL/live_simple_168-99-2 (cmd_controller.execute): start Docker -> 'docker start'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 573 OUT476

✓

SimpleCmdEchoDateBFCL/live_simple_171-99-5 (cmd_controller.execute): today's date -> 'echo %date%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 582 OUT327

✓

SimpleCmdEchoTimeBFCL/live_simple_166-99-0 (cmd_controller.execute): current time -> 'echo %time%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 594 OUT245

×

SimpleCmdListCDriveBFCL/live_simple_170-99-4 (cmd_controller.execute): list C drive -> 'dir C:\'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: (args.command?.ToLower()).AnyOf(["dir c:", "dir c:\\", "dir c:\\\\"])

Failed

IN4 566 OUT300

×

SimpleCmdRunPythonScriptBFCL/live_simple_169-99-3 (cmd_controller.execute): run python script with venv interpreter.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.command?.ToLower()).AnyOf(["d:/playground/pc_contoller/env/scripts/python.exe d:/playground/pc_contoller/test.py"])

Failed

IN4 659 OUT221

✓

SimpleCmdStartCalcBFCL/live_simple_173-99-7 (cmd_controller.execute): open calculator -> 'start calc'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 575 OUT161

✓

SimpleCmdTasklistCodeBFCL/live_simple_167-99-1 (cmd_controller.execute): is VS Code running -> tasklist filter.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 606 OUT664

✓

SimpleProvidersChonburiDateRangeBFCL/live_simple_186-111-0 (get_service_providers): ironing, Chonburi, rating>=4.5, 12/03/2024 16:00-18:00.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 748 OUT1 366

Tool Calling - Simple: Live Services, Weather & Commands (set 11) 3 passed1 failed

75.0%

IN19 689 OUT2 391

✓

SimpleItemsCircleLogosBFCL/live_simple_162-96-1 (get_items): 4 circle-logo copyright items.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 826 OUT296

✓

SimpleItemsLegoMovieBFCL/live_simple_163-96-2 (get_items): 7 Lego-movie copyright items.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 939 OUT409

✓

SimpleItemsSuperheroLogosBFCL/live_simple_161-96-0 (get_items): 12 superhero-logo copyright items.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 055 OUT738

×

SimpleProvidersChonburiNoLateBFCL/live_simple_187-112-0 (get_service_providers): ironing, Chonburi, rating>=4.5, 12/03/2024, never late. NOTE: the prompt gives a time range "16:00 - 18:00", so end_available_date is expected; the 27B occasionally omits it (~1/3 of runs) -> a legit model-reliability miss, not an assertion flaw.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.end_available_date ?? "").AnyOfContains(["2024-03-12", "2024-12-03"])

Failed

IN4 869 OUT948

Tool Calling - Simple: Live Services, Weather & Commands (set 12) 1 passed1 failed

50.0%

IN9 448 OUT1 062

×

SimpleCopyrightCocaColaBFCL/live_simple_164-97-0 (get_copyright_info): Coca-Cola logo, The Coca-Cola Company, confidence 100.0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.copyright_content?.ToLower()).AnyOf(["coca-cola logo"])

Failed

IN4 877 OUT767

✓

SimpleProfessionalCleaningRating2BFCL/live_simple_183-108-0 (getDataForProfessional): cleaning in Bangkok, rating>=2.0 -> avg_rating=2.0, province_id=1, service_id=1.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 571 OUT295

Tool Calling - Simple: Live Services, Weather & Commands (set 13) 3 passed

100.0%

IN13 690 OUT1 237

✓

SimpleMpnSpecs3pak7BFCL/live_simple_191-115-1 (raptor.mpn.specs): identifier=3pak7, MPN, include_images omitted/false.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 559 OUT548

✓

SimpleMpnSpecsAchdImagesBFCL/live_simple_190-115-0 (raptor.mpn.specs): identifier=ACHD, MPN, include_images=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 569 OUT333

✓

SimpleProfessionalCleaningRating4BFCL/live_simple_184-109-0 (getDataForProfessional): cleaning in Bangkok, rating>4 -> service_id=1, province_id=1, rating=4.0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 562 OUT356

Tool Calling - Simple: Live Services, Weather & Commands (set 14) 2 passed

100.0%

IN9 821 OUT809

✓

SimpleExtractChineseNamesBFCL/live_simple_165-98-0 (extractor.extract_information): Chinese text -> [李雷,18],[李丽,21] (sister 3 yrs older).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 942 OUT361

✓

SimpleProfessionalIroningChiangMaiBFCL/live_simple_188-113-0 (getDataForProfessional): ironing in Chiang Mai, Sankampang/Ton Pao, rating 4.5.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 879 OUT448

Tool Calling - Simple: Live Services, Weather & Commands (set 2) 3 passed

100.0%

IN13 110 OUT984

✓

SimpleServiceIdIroningUnitSessionBFCL/live_simple_175-101-0 (get_service_id): ironing -> service_id=2, unit omitted/session. Thai prompt.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 353 OUT405

✓

SimpleWeatherMartinezBFCL/live_simple_194-116-2 (Weather_1_GetWeather): city=Martinez, date=2023-04-25.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 387 OUT237

✓

SimpleWeatherPaloAltoBFCL/live_simple_195-116-3 (Weather_1_GetWeather): city=Palo Alto, date=2023-04-25.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 370 OUT342

Tool Calling - Simple: Live Services, Weather & Commands (set 3) 3 passed

100.0%

IN13 072 OUT1 032

✓

SimpleServiceIdIroningUnitQtyBFCL/live_simple_176-102-0 (get_service_id): ironing -> service_id=2, unit omitted/1. Thai prompt.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 339 OUT415

✓

SimpleWeatherAlamedaWednesdayBFCL/live_simple_196-116-4 (Weather_1_GetWeather): city=Alameda, upcoming Wednesday (today Mon 2023-04-25) -> 2023-04-27.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 369 OUT326

✓

SimpleWeatherStinsonBeachBFCL/live_simple_197-116-5 (Weather_1_GetWeather): city=Stinson Beach, date=2023-04-05.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 364 OUT291

Tool Calling - Simple: Live Services, Weather & Commands (set 4) 4 passed

100.0%

IN17 403 OUT1 186

✓

SimpleServiceIdCleaningBFCL/live_simple_177-103-0 (get_service_id): cleaning -> service_id=1, unit omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 325 OUT324

✓

SimpleServiceIdIroningBFCL/live_simple_178-103-1 (get_service_id): ironing -> service_id=2, unit omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 327 OUT300

✓

SimpleWeatherHealdsburgBFCL/live_simple_198-116-6 (Weather_1_GetWeather): city=Healdsburg, date=2023-03-02.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 380 OUT279

✓

SimpleWeatherMarshallBFCL/live_simple_199-116-7 (Weather_1_GetWeather): city=Marshall (prompt 'Marshall, MN'), date=2023-03-05. NOTE: GT city is bare "Marshall"; model faithfully copies the "Marshall, MN" disambiguator from the prompt -> substring-intent (equivalent-correct).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 371 OUT283

Tool Calling - Simple: Live Services, Weather & Commands (set 5) 3 passed

100.0%

IN13 209 OUT1 010

✓

SimpleServiceIdIroningChiangMaiBFCL/live_simple_179-104-0 (get_service_id): ironing in Chiang Mai -> service_id=2, province_id=2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 374 OUT271

✓

SimpleWeatherCampbellBFCL/live_simple_201-116-9 (Weather_1_GetWeather): city=Campbell, date=2023-03-04.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 431 OUT461

✓

SimpleWeatherFremontBFCL/live_simple_200-116-8 (Weather_1_GetWeather): city=Fremont, date=2023-03-01.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 404 OUT278

Tool Calling - Simple: Live Services, Weather & Commands (set 6) 3 passed

100.0%

IN13 396 OUT923

✓

SimpleServiceIdIroningBangkokRatingBFCL/live_simple_180-105-0 (get_service_id): ironing in Bangkok, rating>=4 -> service_id=2, province_id=1, rating=4.0.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 468 OUT376

✓

SimpleWeatherFosterCityBFCL/live_simple_202-116-10 (Weather_1_GetWeather): city=Foster City, date=2023-04-25.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 461 OUT330

✓

SimpleWeatherWashingtonBFCL/live_simple_203-116-11 (Weather_1_GetWeather): city=Washington (prompt 'Washington, DC'), date=2023-03-01. NOTE: GT city is bare "Washington"; model faithfully copies the "Washington, DC" disambiguator from the prompt -> substring-intent (equivalent-correct).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 467 OUT217

Tool Calling - Simple: Live Services, Weather & Commands (set 7) 3 passed

100.0%

IN13 721 OUT1 327

✓

SimpleServiceIdIroningChatuchakBFCL/live_simple_181-106-0 (get_service_id): ironing, Bangkok(Chatuchak), rating>=4.5 -> service_id=2, province_id omitted/1, district_name=Chatuchak, rating=4.5.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 585 OUT372

✓

SimpleWeatherBerkeleySaturdayBFCL/live_simple_205-116-13 (Weather_1_GetWeather): city=Berkeley, upcoming Saturday (today Tue 2023-04-25) -> 2023-04-29.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 564 OUT520

✓

SimpleWeatherRutherfordBFCL/live_simple_204-116-12 (Weather_1_GetWeather): city=Rutherford (prompt 'Rutherford, NJ'), date=2023-04-22. NOTE: GT city is bare "Rutherford"; model faithfully copies the "Rutherford, NJ" disambiguator from the prompt -> substring-intent (equivalent-correct).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 572 OUT435

Tool Calling - Simple: Live Services, Weather & Commands (set 8) 3 passed

100.0%

IN13 820 OUT1 173

✓

SimpleProvidersKhlongSamWaBFCL/live_simple_182-107-0 (get_service_providers): ironing, Bangkok, Khlong Sam Wa / Sai Kong, rating>=4.5.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 639 OUT539

✓

SimpleWeatherLondonBFCL/live_simple_206-116-14 (Weather_1_GetWeather): city=London (prompt 'London, England'), date=2023-03-05.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 591 OUT378

✓

SimpleWeatherSacramentoBFCL/live_simple_207-116-15 (Weather_1_GetWeather): city=Sacramento, date=2023-04-22.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 590 OUT256

Tool Calling - Simple: Live Services, Weather & Commands (set 9) 5 passed1 failed

83.3%

IN27 839 OUT1 669

✓

SimpleCmdEchoTimeUnitBFCL/live_simple_157-95-14 (cmd_controller.execute): current time -> 'echo %time%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 619 OUT428

×

SimpleCmdListCDriveUnitBFCL/live_simple_156-95-13 (cmd_controller.execute): list C drive -> 'dir C:\'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.command?.ToLower()).AnyOf(["dir c:", "dir c:\\", "dir c:\\\\"])

Failed

IN4 641 OUT195

✓

SimpleCmdStartCameraBFCL/live_simple_160-95-17 (cmd_controller.execute): open camera -> 'start microsoft.windows.camera:'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 614 OUT158

✓

SimpleCmdTaskkillCodeBFCL/live_simple_158-95-15 (cmd_controller.execute): kill VS Code -> 'taskkill /IM code.exe /F'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 614 OUT201

✓

SimpleCmdTimeout10BFCL/live_simple_159-95-16 (cmd_controller.execute): timer 10s -> 'timeout 10'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 634 OUT208

✓

SimpleProvidersChonburiTomorrowBFCL/live_simple_185-110-0 (get_service_providers): ironing, Chonburi, rating>=4.5, tomorrow of 20230929 -> 2023-09-30.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 717 OUT479

Tool Calling - Simple: Live Shell, Classification & APIs (set 1) 3 passed

100.0%

IN13 896 OUT1 926

✓

SimpleClassifyGreetingsAndLinksBFCL/live_simple_105-62-0 (classify): clean_hello=['hello'], faq_link_accounts_start=['link my account','connect my accounts'].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 611 OUT253

✓

SimpleCmdDockerVersionBFCL/live_simple_141-94-0 (cmd_controller.execute): command='docker --version'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 576 OUT364

✓

SimpleReqNodesPekAciBFCL/live_simple_128-83-0 (requests.get): nodes list url with ip 192.168.11.33, fabricName='PEK-ACI'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 709 OUT1 309

Tool Calling - Simple: Live Shell, Classification & APIs (set 10) 4 passed

100.0%

IN19 605 OUT1 164

✓

SimpleCmdShutdownBFCL/live_simple_150-95-7 (cmd_controller.execute): shutdown the pc via 'shutdown /s /t 0'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 866 OUT149

✓

SimpleProcessImageMoondreamBFCL/live_simple_122-78-0 (process_data): image.png + long question + model 'vikhyatk/moondream2'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 978 OUT577

✓

SimpleReqDataLimit50BFCL/live_simple_135-88-0 (requests.get): data endpoint url, params limit=50.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 897 OUT283

✓

SimpleSumNumbersListBFCL/live_simple_116-72-0 (sum_numbers): numbers_list=[133, 34].1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 864 OUT155

Tool Calling - Simple: Live Shell, Classification & APIs (set 11) 4 passed

100.0%

IN20 023 OUT1 467

✓

SimpleCmdWifiDisconnectBFCL/live_simple_151-95-8 (cmd_controller.execute): disconnect wifi via 'netsh wlan disconnect'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 893 OUT146

✓

SimplePipelineFurnitureImageBFCL/live_simple_123-79-0 (pipeline): analyze thewowstyle image with furniture/minimalist question.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 169 OUT762

✓

SimpleReqTopologySalesAppBFCL/live_simple_136-89-0 (requests.get): topology url ip 192.168.1.1, filter accountName:AcmeCorp AND applicationName:SalesApp.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 050 OUT313

✓

SimpleTemperatureHyderabadBFCL/live_simple_118-74-0 (get_temperature): units=Celsius, location='Hyderabad, India', time omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 911 OUT246

Tool Calling - Simple: Live Shell, Classification & APIs (set 12) 4 passed

100.0%

IN21 311 OUT1 790

✓

SimpleCmdDirDAndEchoBFCL/live_simple_152-95-9 (cmd_controller.execute): 'dir D:\ && echo testing.txt'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 272 OUT550

✓

SimpleReqAnomaliesSummaryBFCL/live_simple_139-92-0 (requests.get): anomalies summary url ip 192.168.1.1, params fabric='network222'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 399 OUT474

✓

SimpleRescheduleAlice8pmBFCL/live_simple_137-90-0 (reschedule): identifier='Alice-One-one-One', 2023-11-01T20:00:00, Europe/London.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 306 OUT267

✓

SimpleWeatherForecastParisBFCL/live_simple_120-76-0 (weather.forecast): Paris, France 2023-04-03..05, Celsius, defaults omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 334 OUT499

Tool Calling - Simple: Live Shell, Classification & APIs (set 13) 3 passed

100.0%

IN14 709 OUT1 690

✓

SimpleCmdDelPuttyBFCL/live_simple_153-95-10 (cmd_controller.execute): remove putty shortcut via del with quoted path.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 818 OUT223

✓

SimpleDynamicPricingSaaSBFCL/live_simple_113-69-0 (calculate_dynamic_pricing): geolocation='34.0522, -118.2437', base=100, min=90.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 038 OUT879

✓

SimpleRescheduleBob4pmBFCL/live_simple_138-91-0 (reschedule): identifier='Bob-123', 2023-11-01T16:00:00, Europe/London.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 853 OUT588

Tool Calling - Simple: Live Shell, Classification & APIs (set 14) 3 passed

100.0%

IN14 094 OUT895

✓

SimpleCmdStartYoutubeBFCL/live_simple_154-95-11 (cmd_controller.execute): open youtube rickroll via start command.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 711 OUT226

✓

SimpleGetTicketsCompanyOpenBFCL/live_simple_124-80-0 (get_tickets): customer (company name in prompt), status=open, limit=10.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 696 OUT406

✓

SimpleGreetPartnerJeffBFCL/live_simple_140-93-0 (greet_partner): user='Emily', partner='Jeff', relationship='husband'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 687 OUT263

Tool Calling - Simple: Live Shell, Classification & APIs (set 15) 2 passed1 failed

66.7%

IN14 469 OUT1 146

✓

SimpleCalculateTaxNewYorkBFCL/live_simple_104-61-2 (calculate_tax): purchase_amount=250, state='NY', all optionals omitted/default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 834 OUT612

✓

SimpleCmdDateBFCL/live_simple_155-95-12 (cmd_controller.execute): what is the date via 'date /t' or 'echo %date%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 805 OUT381

×

SimpleConcatJohnDoeBFCL/live_simple_125-81-0 (concat_strings): strings=['John','Doe'], separator=' '. EXPECTED MODEL MISS: 'combine ... into a full name' requires separator=' ' (John Doe). The 27B tends to omit separator (=> 'JohnDoe'); that is a legit model failure of the measured competence, so the ' ' assertion stays strict.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.separator.AnyOf([" "]) Values: args.separator = null

Failed

IN4 830 OUT153

Tool Calling - Simple: Live Shell, Classification & APIs (set 2) 3 passed

100.0%

IN16 106 OUT2 445

✓

SimpleCmdDirDesktopBFCL/live_simple_142-94-1 (cmd_controller.execute): command='dir Desktop'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 041 OUT208

✓

SimpleRecordBankNineIntentsBFCL/live_simple_106-63-0 (record): classify 20 bank queries into 9 intents (2 buckets empty).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 963 OUT1 666

✓

SimpleReqNodesFabEdBFCL/live_simple_129-83-1 (requests.get): nodes list url with ip 3.321.3232.2, fabricName='fab-ed'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 102 OUT571

Tool Calling - Simple: Live Shell, Classification & APIs (set 3) 4 passed

100.0%

IN21 436 OUT1 645

✓

SimpleCmdDockerPsBFCL/live_simple_143-95-0 (cmd_controller.execute): command='docker ps'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 217 OUT210

✓

SimpleRecordTelephoneHoursAccountsBFCL/live_simple_107-64-0 (record): classify 7 queries into telephone-banking / bank-hours / accounts.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 617 OUT793

✓

SimpleTelemetryGlobalFabricStatusBFCL/live_simple_126-82-0 (telemetry.flowrules.interfaceInfo.get): fabric='Global-Fabric', node=1200, pod=3, ethernet, status.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 299 OUT398

✓

SimpleUpdateProfileEmailAgeBFCL/live_simple_114-70-0 (update_user_profile): user_id=12345, email+age set, bio omitted, notify default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 303 OUT244

Tool Calling - Simple: Live Shell, Classification & APIs (set 4) 3 passed1 failed

75.0%

IN20 858 OUT2 494

✓

SimpleCmdKillFirefoxBFCL/live_simple_144-95-1 (cmd_controller.execute): close firefox via taskkill.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 081 OUT213

×

SimpleRecordMoneyTransactionsCreditBFCL/live_simple_108-65-0 (record): classify 10 queries into money-movement / transactions / credit-limit / link-accounts. EXPECTED MODEL MISS: the 27B classifies 'show me all outgoing wire transfers' as a transaction-view (get_transactions_start) rather than money_movement_start per GT, so the get_transactions count comes out as 5 instead of 4. GT bucketing kept strict; this is a legit model classification failure.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: args.money_movement_start.Count == 1 Values: args.money_movement_start.Count = 0

Failed

IN5 549 OUT1 795

✓

SimpleReverseSayHiBFCL/live_simple_117-73-0 (reverse_input): input_value='say hi'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 069 OUT192

✓

SimpleTelemetryTestDeStatusBFCL/live_simple_127-82-1 (telemetry.flowrules.interfaceInfo.get): fabric='test-de', node=5, pod=1, ethernet, status.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 159 OUT294

Tool Calling - Simple: Live Shell, Classification & APIs (set 5) 4 passed

100.0%

IN20 873 OUT1 968

✓

SimpleCmdDirCDriveBFCL/live_simple_145-95-2 (cmd_controller.execute): list C drive => 'dir C:\'. EXPECTED MODEL MISS: for the terse/typo'd 'LIST C DRIIVE' the 27B currently returns 'dir d:' — the wrong drive. The accepted-command list is broadened with the no-backslash 'dir c:' variant for fairness, but a wrong drive letter is a legit model failure.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 106 OUT205

✓

SimplePrimeMinistersThreeBFCL/live_simple_119-75-0 (GetPrimeMinisters): countries=[Australia, Canada, India].1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 139 OUT345

✓

SimpleRecordAutoWithdrawIncomeBFCL/live_simple_109-66-0 (record): classify 5 loan/withdraw queries; outofscope empty.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 357 OUT564

✓

SimpleReqInterfaceStatsNetwork1BFCL/live_simple_130-84-0 (requests.get): interfaceInfo stats, ip 192.120.45.67, Network1/node 12/pod 10/ethernet/statistics.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 271 OUT854

Tool Calling - Simple: Live Shell, Classification & APIs (set 6) 4 passed

100.0%

IN21 534 OUT1 904

✓

SimpleCmdStartHuggingfaceBFCL/live_simple_146-95-3 (cmd_controller.execute): open huggingface via start command.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 292 OUT179

✓

SimpleGenerateSalaryChartBFCL/live_simple_121-77-0 (generate_chart): 5 job labels and salaries in order.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 446 OUT407

✓

SimpleRecordAtmLocationOnlyBFCL/live_simple_110-67-0 (record): single query 'Need ATM location' -> atm_finder_start only.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 291 OUT365

✓

SimpleReqInterfaceSpeedNetwork1BFCL/live_simple_131-84-1 (requests.get): interfaceInfo Speed, ip 192.120.45.67, Network1/node 12/pod 10/ethernet/Speed.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 505 OUT953

Tool Calling - Simple: Live Shell, Classification & APIs (set 7) 3 passed

100.0%

IN14 937 OUT1 327

✓

SimpleCmdKillTimerBFCL/live_simple_147-95-4 (cmd_controller.execute): remove timer.exe via taskkill.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 865 OUT245

✓

SimpleRecordBalanceAndAtmBFCL/live_simple_111-67-1 (record): 10 queries -> 8 balance + 2 atm; other buckets empty.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 200 OUT808

✓

SimpleReqDeviceConnectorVersionsBFCL/live_simple_132-85-0 (requests.get): device connector versions JSON url, no params.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 872 OUT274

Tool Calling - Simple: Live Shell, Classification & APIs (set 8) 2 passed1 failed

66.7%

IN14 814 OUT1 683

×

SimpleCmdWhatFileInCBFCL/live_simple_148-95-5 (cmd_controller.execute): 'what is the file in c' => 'dir C:\'. EXPECTED MODEL MISS: the 27B mis-selects the 'record' classifier (dropping the request into its 'outofscope' bucket) instead of cmd_controller_execute. record is a legitimate distractor here (not a looping universal collider), so this stays as a real tool-selection failure — no isolation needed.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: args.command.ToLower().AnyOf(["dir c:\\", "dir c:\\\\"])

Failed

IN4 894 OUT880

✓

SimpleRecordWeatherOutOfScopeBFCL/live_simple_112-68-0 (record): single query 'what is the weather like' -> outofscope only.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 958 OUT454

✓

SimpleReqRecommendationsDacBFCL/live_simple_133-86-0 (requests.get): recommendations url, params advisoryId='dac'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 962 OUT349

Tool Calling - Simple: Live Shell, Classification & APIs (set 9) 3 passed

100.0%

IN13 871 OUT575

✓

SimpleCalculateSum133And34BFCL/live_simple_115-71-0 (calculate_sum): number1=133, number2=34.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 607 OUT177

✓

SimpleCmdEchoHiBFCL/live_simple_149-95-6 (cmd_controller.execute): say hi via echo command.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 581 OUT184

✓

SimpleReqInsightsGroupsBFCL/live_simple_134-87-0 (requests.get): insights groups url, params limit=20, sort='desc'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 683 OUT214

Tool Calling - Simple: Math, Physics & Geometry (set 1) 12 passed

100.0%

IN88 604 OUT4 520

✓

SimpleAlgebraQuadraticRootsBFCL/simple_python_3 (algebra.quadratic_roots): a=1, b=-3, c=2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 395 OUT315

✓

SimpleAreaUnderCurveXSquaredBFCL/simple_python_13 (calculate_area_under_curve): function y=x^2, interval [1,3], method omitted/trapezoidal.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 392 OUT462

✓

SimpleCircleCircumferenceInchesBFCL/simple_python_7 (calculate_circumference): radius=4, unit inches/in.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 376 OUT254

✓

SimpleDerivativeCubicPolyBFCL/simple_python_14 (calculate_derivative): function 3x^2+2x-1, x_value omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 383 OUT261

✓

SimpleElectricFieldFromChargeBFCL/simple_python_39 (calculate_electric_field): charge=2, distance=3, permitivity omitted/8.854e-12.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 384 OUT447

✓

SimpleFinalVelocityFromDistanceBFCL/simple_python_30 (kinematics.final_velocity_from_distance): acceleration=4, distance=300, initial_velocity omitted/0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 411 OUT592

✓

SimpleFinalVelocityFromHeightBFCL/simple_python_25 (calculate_final_velocity): height=150, initial_velocity omitted/0, gravity omitted/9.81.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 383 OUT345

✓

SimpleGeometryAreaCircleBFCL/simple_python_8 (geometry.area_circle): radius=10, units omitted or 'meters'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 365 OUT254

✓

SimpleGetPrimeFactors450BFCL/simple_python_17 (get_prime_factors): number=450, formatted omitted or true.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 355 OUT275

✓

SimpleMagneticFieldLoopBFCL/simple_python_40 (calculate_magnetic_field): current=5, radius=4, permeability omitted/12.57e10.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 396 OUT769

✓

SimpleMathGcd40And50BFCL/simple_python_19 (math.gcd): num1=40, num2=50.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 379 OUT241

✓

SimpleTriangleAreaWithUnitBFCL/simple_python_0 (calculate_triangle_area): base=10, height=5, unit omitted or 'units'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 385 OUT305

Tool Calling - Simple: Math, Physics & Geometry (set 2) 12 passed

100.0%

IN88 306 OUT4 328

✓

SimpleAverageVelocityBFCL/simple_python_26 (calculate_velocity): distance=50, duration=2, unit omitted or 'km/h'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 359 OUT202

✓

SimpleCalculusDerivativeAtValueBFCL/simple_python_16 (calculus.derivative): function 2x^2, value=1, function_variable omitted or 'x'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 350 OUT258

✓

SimpleElectricFieldStrengthBFCL/simple_python_44 (calculate_electric_field_strength): charge=0.01, distance=4, medium omitted or 'vacuum'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 360 OUT450

✓

SimpleFinalVelocityConstAccelBFCL/simple_python_31 (calculate_final_velocity): initial_velocity=0, acceleration=9.8, time=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 397 OUT299

✓

SimpleGeometryCalculateAreaCircleBFCL/simple_python_9 (geometry.calculate_area_circle): radius=5, unit omitted or 'units'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 334 OUT235

✓

SimpleGeometryCircumferenceBFCL/simple_python_12 (geometry.circumference): radius=3, units omitted or 'cm'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 327 OUT275

✓

SimpleIntegrateCubeSimpsonBFCL/simple_python_15 (integrate): function x^3, start_x=-2, end_x=3, method simpson.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 392 OUT620

✓

SimpleMagneticFieldStrengthWireBFCL/simple_python_43 (calculate_magnetic_field_strength): current=20, distance=10, permeability omitted/12.57e-7.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 367 OUT447

✓

SimpleMathHcfBFCL/simple_python_20 (math.hcf): number1=36, number2=24.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 345 OUT355

✓

SimpleNumberAnalysisPrimeFactorsBFCL/simple_python_18 (number_analysis.prime_factors): number=123456.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 341 OUT542

✓

SimpleRightTriangleAreaCmBFCL/simple_python_10 (calculate_area): base=6, height=10, unit omitted or 'cm'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 376 OUT226

✓

SimpleSolveQuadraticEquation265BFCL/simple_python_4 (solve_quadratic_equation): a=2, b=6, c=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 358 OUT419

Tool Calling - Simple: Math, Physics & Geometry (set 3) 9 passed

100.0%

IN60 218 OUT3 769

✓

SimpleBoilingMeltingPointsBFCL/simple_python_47 (get_boiling_melting_points): substance='water', sea_level=5000.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 684 OUT927

✓

SimpleDisplacementBFCL/simple_python_28 (calculate_displacement): initial_velocity=10, time=5, acceleration=9.8.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 702 OUT325

✓

SimpleElectrostaticPotentialBFCL/simple_python_38 (calculate_electrostatic_potential): charge1=1e-9, charge2=2e-9, distance=0.05, constant omitted/8.99e9.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 716 OUT553

✓

SimpleFinalSpeedFromHeightBFCL/simple_python_32 (calculate_final_speed): initial_velocity=0, height=100, gravity omitted or 9.8.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 672 OUT273

✓

SimpleNumberTheoryGcdBFCL/simple_python_21 (number_theory.gcd): number1=36, number2=48.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 685 OUT219

✓

SimplePrimeFactorizeDictionaryBFCL/simple_python_23 (prime_factorize): number=60, return_type='dictionary'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 676 OUT275

✓

SimpleRouteEstimateTimeStopsBFCL/simple_python_37 (route.estimate_time): SF -> LA, stops {Santa Barbara, Monterey} in either order.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 702 OUT517

✓

SimpleSolveQuadraticAllRootsBFCL/simple_python_5 (solve_quadratic): a=3, b=-11, c=-4, root_type='all' (user asked for all roots).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 705 OUT440

✓

SimpleTriangleAreaNoUnitBFCL/simple_python_11 (calculate_triangle_area): base=10, height=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 676 OUT240

Tool Calling - Simple: Math, Physics & Geometry (set 4) 9 passed

100.0%

IN59 747 OUT3 758

✓

SimpleCalculateDensityBFCL/simple_python_48 (calculate_density): mass=45, volume=15, unit omitted or 'kg/m^3'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 640 OUT208

✓

SimpleElectromagneticForceBFCL/simple_python_41 (electromagnetic_force): charge1=5, charge2=7, distance=3, medium_permittivity omitted/8.854e-12.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 642 OUT603

✓

SimpleFinalSpeedFreeFallTimeBFCL/simple_python_29 (calculate_final_speed): initial_speed omitted/0, time=5, gravity omitted/-9.81.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 636 OUT544

✓

SimpleMathFactorial5BFCL/simple_python_1 (math.factorial): number=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 600 OUT141

✓

SimpleMathGcd12And15BFCL/simple_python_22 (math.gcd): num1=12, num2=15.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 633 OUT193

✓

SimpleShortestDrivingDistanceBFCL/simple_python_36 (get_shortest_driving_distance): origin=New York City, destination=Washington D.C., unit omitted/km.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 625 OUT516

✓

SimpleSolveQuadraticRoots253BFCL/simple_python_6 (solve_quadratic): a=2, b=5, c=3.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 638 OUT256

✓

SimpleThermoEnergyVaporizationBFCL/simple_python_45 (thermo.calculate_energy): mass=100, phase_transition=vaporization, substance omitted/water.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 670 OUT576

✓

SimpleTravelItineraryTokyoBFCL/simple_python_34 (travel_itinerary_generator): destination=Tokyo, days=7, daily_budget=100, exploration_type=nature.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 663 OUT721

Tool Calling - Simple: Math, Physics & Geometry (set 5) 8 passed

100.0%

IN51 550 OUT2 795

✓

SimpleAbsolutePressureBFCL/simple_python_49 (calc_absolute_pressure): atm_pressure=1, gauge_pressure=2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 437 OUT376

✓

SimpleFinalTemperatureMixBFCL/simple_python_46 (calculate_final_temperature): mass1=20, temperature1=30, mass2=15, temperature2=60, shc omitted/4.2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 475 OUT409

✓

SimpleFinalVelocityAccelTimeBFCL/simple_python_27 (final_velocity): initial_velocity=10, acceleration=2, time=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 469 OUT232

✓

SimpleGetDirectionsFastestBFCL/simple_python_33 (get_directions): start=Sydney, end=Melbourne, route_type omitted or 'fastest'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 422 OUT284

✓

SimpleMathGcd12And18BFCL/simple_python_24 (math.gcd): num1=12, num2=18.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 423 OUT211

✓

SimpleMathHypotenuseBFCL/simple_python_2 (math.hypot): x=4, y=5, z omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 433 OUT281

✓

SimpleResonantFrequencyLCBFCL/simple_python_42 (calculate_resonant_frequency): inductance=0.05 (50mH), capacitance=0.0001 (100uF), round_off omitted/2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 460 OUT633

✓

SimpleVeganRestaurantFindBFCL/simple_python_35 (vegan_restaurant.find_nearby): location='New York, NY', operating_hours=23 (11 PM).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 431 OUT369

Tool Calling - Simple: Science, Data & Biology (set 1) 13 passed

100.0%

IN107 010 OUT8 137

✓

SimpleArraySortBFCL/simple_python_87 (array_sort): list=[5,3,4,1,2], order=ascending.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 213 OUT203

✓

SimpleAverageTemperatureBFCL/simple_python_78 (average_temperature): location=Austin, days=3, temp_unit=Celsius.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 197 OUT304

✓

SimpleBacteriaEvolutionRateBFCL/simple_python_74 (calculate_bacteria_evolution_rate): start=5000, dup_freq=1, duration=6, gen_time omitted or 20.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 230 OUT585

✓

SimpleBmiKgCmBFCL/simple_python_84 (calculate_bmi): weight=85, height=180, unit omitted or 'metric'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 216 OUT337

✓

SimpleCreateHistogramBFCL/simple_python_79 (create_histogram): data=[85,90,88,92,86,89,91], bins=5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 273 OUT1 037

✓

SimpleCreatureFitnessBFCL/simple_python_72 (calculate_fitness): trait_values=[0.8,0.7], trait_contributions=[0.4,0.6].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 276 OUT322

✓

SimpleDbFetchStudentsBFCL/simple_python_89 (db_fetch_records): database=StudentDB, table=students, conditions={department:Science, school:Bluebird High School}, fetch_limit omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 227 OUT541

✓

SimpleEntropyChangeSubstanceBFCL/simple_python_50 (entropy_change.calculate): substance=ice, mass=1, init=0, final=100, pressure omitted or 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 265 OUT1 639

✓

SimpleGenerateDnaSequenceBFCL/simple_python_71 (generate_DNA_sequence): length=100, preferences=[G,C] in either order.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 216 OUT320

✓

SimpleGpsCoordinateDistanceBFCL/simple_python_83 (calculate_distance): coord1=[33.4484,-112.074], coord2=[34.0522,-118.2437], unit=miles.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 305 OUT1 818

✓

SimpleImdbMoviesByActorBFCL/simple_python_92 (imdb.find_movies_by_actor): actor='Leonardo DiCaprio', year=2010, category omitted or 'all'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 210 OUT269

✓

SimplePopulationProjectionBFCL/simple_python_73 (population_projections): country=United States/USA, years=20, growth_rate omitted or 1.2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 193 OUT400

✓

SimpleRestaurantFindNearbyBFCL/simple_python_77 (restaurant.find_nearby): location='Los Angeles, CA', dietary_preference=[Vegan].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 189 OUT362

Tool Calling - Simple: Science, Data & Biology (set 2) 13 passed

100.0%

IN106 627 OUT5 141

✓

SimpleBmiMetersBFCL/simple_python_88 (calculate_BMI): weight_kg=70, height_m=1.75.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN8 202 OUT258

✓

SimpleClockAngleBFCL/simple_python_98 (calculate_clock_angle): hours=6, minutes=30, round_to omitted or 2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 186 OUT318

✓

SimpleDatabaseQueryConditionsBFCL/simple_python_96 (database.query): table=user, conditions=[{age,>,25},{job,=,engineer}].1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 225 OUT393

✓

SimpleElephantPopulationBFCL/simple_python_75 (elephant_population_estimate): current=35000, growth_rate=0.015, years=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN8 226 OUT579

✓

SimpleEntropyChangeIsothermalBFCL/simple_python_51 (calculate_entropy_change): initial_temp=300, final_temp=400, heat_capacity=5, isothermal omitted or true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 234 OUT546

✓

SimpleEvolutionPredictionDarwinBFCL/simple_python_76 (prediction.evolution): species=Homo Sapiens, years=50, model=Darwin.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 200 OUT372

✓

SimpleFindRestaurantsThaiBFCL/simple_python_80 (find_restaurants): location=Manhattan variants, food_type=Thai, number=5, dietary_requirements=[vegan].1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 208 OUT496

✓

SimpleGeoDistanceCitiesBFCL/simple_python_85 (geo_distance.calculate): start='Boston, MA', end='Washington, D.C.', units omitted or 'miles'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 197 OUT796

✓

SimpleMathFactorialBFCL/simple_python_97 (math.factorial): number=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN8 149 OUT163

✓

SimplePlotSineWaveBFCL/simple_python_99 (plot_sine_wave): start=0.0, end=6.2832, frequency=5, amplitude omitted or 1, phase_shift omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 197 OUT287

✓

SimpleTheaterMovieReleasesBFCL/simple_python_93 (get_theater_movie_releases): location=LA, timeframe=7, format=IMAX.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 191 OUT463

✓

SimpleTriangleAreaMetersBFCL/simple_python_95 (calc_area_triangle): base=5, height=3.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN8 175 OUT193

✓

SimpleUpdateUserInfoBFCL/simple_python_94 (update_user_info): user_id=43523, update_info={name:John Doe, email:johndoe@email.com}, database omitted or CustomerInfo. NOTE: single-call contract kept intentionally; the model very occasionally emits a spurious second call (args are correct ~3/3 sampled runs), a model-reliability signal rather than a test defect.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 237 OUT277

Tool Calling - Simple: Science, Data & Biology (set 3) 12 passed

100.0%

IN91 470 OUT7 431

✓

SimpleCalculateAverageBFCL/simple_python_82 (calculate_average): numbers=[12,15,18,20,21,26,30].1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 660 OUT402

✓

SimpleCellDensityBFCL/simple_python_57 (calculate_cell_density): optical_density=0.6, dilution=5, calibration_factor omitted or 1e9.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 632 OUT292

✓

SimpleCellInfoHumanBFCL/simple_python_55 (biology.get_cell_info): cell_type=human, detailed=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 596 OUT1 533

✓

SimpleCellProteinsMembraneBFCL/simple_python_56 (cellbio.get_proteins): cell_compartment=plasma membrane, include_description omitted/true/false.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 597 OUT728

✓

SimpleCityDistanceTrainBFCL/simple_python_86 (city_distance.find_shortest): start=New York, end=Los Angeles, transportation=train, allow_transfer=true.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 640 OUT515

✓

SimpleEmployeeFetchDataBFCL/simple_python_90 (employee.fetch_data): company='ABC Ltd.', employee_id=345, data_field=[Personal Info, Job History].1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 649 OUT281

✓

SimpleFetchDnaSequenceBFCL/simple_python_53 (fetch_DNA_sequence): DNA_id=DNA123, format omitted or 'fasta', upstream omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 599 OUT233

✓

SimpleGetRestaurantSushiBFCL/simple_python_91 (get_restaurant): cuisine=sushi, location=Boston, condition=open on Sundays.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 614 OUT390

✓

SimpleHeatCapacityAirBFCL/simple_python_52 (calc_heat_capacity): temp=298, volume=10, gas omitted or 'air'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 643 OUT1 805

✓

SimpleMolecularWeightBFCL/simple_python_59 (calculate_molecular_weight): compound=C6H12O6, to_unit=grams/mole or g/mol.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 626 OUT294

✓

SimpleOrganelleFunctionLookupBFCL/simple_python_58 (cell_biology.function_lookup): molecule=ATP synthase, organelle=mitochondria, specific_function=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 615 OUT640

✓

SimpleProteinSequenceBFCL/simple_python_54 (get_protein_sequence): gene=BRCA1, species omitted or 'Homo sapiens'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 599 OUT318

Tool Calling - Simple: Science, Data & Biology (set 4) 12 passed

100.0%

IN94 128 OUT6 245

✓

SimpleAnalyzeDnaSequenceBFCL/simple_python_62 (analyze_dna_sequence): sequence + reference_sequence given, mutation_type omitted or 'substitution'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 900 OUT662

✓

SimpleDiabetesPredictionBFCL/simple_python_61 (diabetes_prediction): weight=150, height=70 (5ft10in), activity_level=lightly active.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 861 OUT542

✓

SimpleFastestRouteAvoidTollsBFCL/simple_python_81 (map_routing.fastest_route): start=San Francisco/SF, end=Los Angeles/LA, avoid_tolls=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 830 OUT684

✓

SimpleForestGrowthForecastBFCL/simple_python_68 (forest_growth_forecast): location=Yellowstone National Park, years=5, include_human_impact=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 837 OUT537

✓

SimpleGeneticSimilarityBFCL/simple_python_63 (genetics.calculate_similarity): species1=Human, species2=Chimp/Chimpanzee, format omitted or 'percentage'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 820 OUT633

✓

SimpleGenotypeFrequencyBFCL/simple_python_64 (calculate_genotype_frequency): allele_frequency=0.3, genotype=AA.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 839 OUT341

✓

SimpleIdentifyBirdBFCL/simple_python_67 (identify_bird): color=green, habitat=forest, size=small.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 813 OUT368

✓

SimpleMutationTypeFindBFCL/simple_python_60 (mutation_type.find): snp_id=rs6034464, species omitted or 'Homo sapiens'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 843 OUT345

✓

SimplePopulationDensityBFCL/simple_python_65 (calculate_density): country=Brazil, year='2022', population=213000000, land_area=8500000.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 886 OUT408

✓

SimplePrecipitationStatsBFCL/simple_python_66 (ecology_data.precipitation_stats): location=Amazon rainforest, time_frame=six_months.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 828 OUT395

✓

SimpleTurtlePopulationBFCL/simple_python_69 (ecology.get_turtle_population): location=Mississippi river, year=2020, species=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 836 OUT898

✓

SimpleVehicleEmissionBFCL/simple_python_70 (calculate_vehicle_emission): vehicle_type=gas, miles_driven=1500, emission_factor omitted or 355.48.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 835 OUT432

Tool Calling - Simple: Sports & Games (set 1) 12 passed

100.0%

IN91 964 OUT6 312

✓

SimpleBattingAverageComputeBFCL/simple_python_302 (calculate_batting_average): hits=180, at_bats=600, decimal_places=3 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 667 OUT204

✓

SimpleBoardGameInfoTicketBFCL/simple_python_328 (boardgame.get_info): Ticket to Ride, {rating, player count} either order, language omitted/English.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 636 OUT684

✓

SimpleFindCardInDeckBFCL/simple_python_335 (find_card_in_deck): rank Queen, suit Hearts, deck omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 629 OUT133

✓

SimpleGameListByYearBFCL/simple_python_342 (game_list.get_games): year 2019, multiplayer true, ESRB Everyone.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 650 OUT244

✓

SimpleGameWinnerByDateBFCL/simple_python_307 (game_result.get_winner): Lakers/Clippers either order, date 2021-01-28, venue omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 648 OUT474

✓

SimpleHighestGameScoreBFCL/simple_python_349 (game_score.highest): Overwatch, PC, region Global or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 641 OUT424

✓

SimpleMatchScheduleBFCL/simple_python_308 (sports.match_schedule): Man United, 5 matches, English Premier League or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 646 OUT331

✓

SimpleNoteDurationBFCL/simple_python_300 (music.calculate_note_duration): 440/880 Hz, tempo omitted or 120.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 648 OUT871

✓

SimplePokerGameWinnerMultiBFCL/simple_python_337 (poker_game_winner): 4 players + per-player hands dict, type omitted/Texas Holdem.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 883 OUT658

✓

SimpleSoccerPlayerStatsBFCL/simple_python_303 (soccer_stat.get_player_stats): Cristiano Ronaldo, 2019-2020, league omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 646 OUT232

✓

SimpleSportRankingPlayerBFCL/simple_python_316 (get_sport_ranking): Tennis, Serena Williams, gender omitted/all/female.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 623 OUT435

✓

SimpleTeamRankRegularBFCL/simple_python_317 (get_team_rank): LA Lakers, NBA, 2021, type regular.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 647 OUT1 622

Tool Calling - Simple: Sports & Games (set 2) 11 passed

100.0%

IN78 484 OUT4 760

✓

SimpleBoardGameInfoCatanBFCL/simple_python_330 (board_game_info): Catan, {average_review_rating, age_range}.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 132 OUT528

✓

SimpleCricketBattingScoreBFCL/simple_python_306 (average_batting_score): Virat Kohli, 10 matches, T20 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 125 OUT210

✓

SimpleFifaTeamRankingBFCL/simple_python_318 (get_team_ranking): Germany, 2021, gender men or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 132 OUT438

✓

SimpleMatchResultsTwoTeamsBFCL/simple_python_325 (sports.match_results): Chicago Bulls vs Los Angeles Lakers, season omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 110 OUT397

✓

SimplePokerDetermineWinnerTwoBFCL/simple_python_340 (card_games.poker_determine_winner): John vs Mike with 5-card hands (suit symbols).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 258 OUT625

✓

SimpleShuffleAndDrawBFCL/simple_python_336 (cards.shuffle_and_draw): num_cards 3.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 104 OUT222

✓

SimpleSoccerPerformanceBFCL/simple_python_305 (sports_stats.get_performance): Messi, La Liga, 2020-2021, {goals scored, assists made}.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 168 OUT371

✓

SimpleTeamScheduleGamesBFCL/simple_python_327 (sports_team.get_schedule): Man United, 6 games, Premier League, location omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 128 OUT551

✓

SimpleThirdChordBFCL/simple_python_301 (get_third_chord): key C, type major or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 102 OUT687

✓

SimpleTopPlayerBySportBFCL/simple_python_323 (sports_ranking.get_top_player): tennis, gender women.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 105 OUT492

✓

SimpleVideoGamePlayerStatsBFCL/simple_python_343 (game_stats.fetch_player_statistics): Zelda, user Sam, platform Switch.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 120 OUT239

Tool Calling - Simple: Sports & Games (set 3) 9 passed

100.0%

IN58 070 OUT3 346

✓

SimpleCardDrawProbabilityBFCL/simple_python_338 (card_game_probability.calculate): total 52, desired 13 (hearts), cards_drawn 1 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 454 OUT299

✓

SimpleCelebrityNetWorthBFCL/simple_python_313 (celebrity_net_worth.get): Messi, currency EUR/euro.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 448 OUT441

✓

SimpleDefenseRankingBFCL/simple_python_315 (get_defense_ranking): season 2021, top 1 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 455 OUT515

✓

SimpleFindAthleteBFCL/simple_python_311 (sports_db.find_athlete): Lebron James, sport Basketball, team omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 440 OUT290

✓

SimpleGameItemStatsBFCL/simple_python_344 (get_game_item_stats): BotW, Guardian Sword+, stat power/power rating.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 483 OUT353

✓

SimpleMonopolyOddsBFCL/simple_python_329 (monopoly_odds_calculator): number 7, dice_number 2, dice_faces 6 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 452 OUT397

✓

SimpleNflPlayerRecordBFCL/simple_python_309 (nfl_data.player_record): Tom Brady, 2020, team omitted or Tampa Bay Buccaneers.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 454 OUT310

✓

SimpleSportsRankingSeasonIntBFCL/simple_python_319 (sports_ranking): Man United, Premier League, season omitted or 2023.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 438 OUT328

✓

SimpleTeamLatestScoreBFCL/simple_python_324 (team_score.get_latest): Los Angeles Lakers, include_opponent true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 446 OUT413

Tool Calling - Simple: Sports & Games (set 4) 7 passed

100.0%

IN41 841 OUT3 300

✓

SimpleCareerStatsBFCL/simple_python_310 (get_career_stats): LeBron James, team omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 954 OUT588

✓

SimpleCelebrityAchievementsBFCL/simple_python_314 (sports_celebrity.get_major_achievements): Messi, sports Football/Soccer or omitted, team all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 964 OUT533

✓

SimpleChessTopPlayersBFCL/simple_python_331 (board_game.chess.get_top_players): New York, min rating 2300, number_of_players 10 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 979 OUT314

✓

SimpleDeckOddsSuitBFCL/simple_python_341 (deck_of_cards.odds): suit hearts, deck_type without_joker or normal.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 971 OUT340

✓

SimpleGameValuationBFCL/simple_python_345 (game_valuation): Super Mario Bros., 1985, condition Like New/New.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 991 OUT656

✓

SimpleTeamPositionDetailedBFCL/simple_python_320 (sports_ranking.get_team_position): Golden State Warriors, 2022-2023, detailed true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 007 OUT579

✓

SimpleTeamScoreWithStatsBFCL/simple_python_326 (get_team_score): Los Angeles Lakers, NBA, include_player_stats any.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 975 OUT290

Tool Calling - Simple: Sports & Games (set 5) 7 passed

100.0%

IN40 523 OUT3 198

✓

SimpleChessRatingBFCL/simple_python_332 (chess.rating): Magnus Carlsen, variant classical or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 772 OUT331

✓

SimpleCollectablesInSeasonBFCL/simple_python_346 (get_collectables_in_season): Animal Crossing: New Horizons, Spring, item_type all or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 795 OUT439

✓

SimpleDetailedWeatherForecastBFCL/simple_python_333 (detailed_weather_forecast): London UK, 3 days, {high/low temp, humidity, precipitation}.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 830 OUT652

✓

SimpleFullHouseProbabilityBFCL/simple_python_339 (poker_probability.full_house): defaults deck_size 52, hand_size 5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 773 OUT770

✓

SimplePlayerStatisticYearBFCL/simple_python_312 (player_statistic): Ronaldo, year 2021, team omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 779 OUT320

✓

SimpleSoccerLastMatchBFCL/simple_python_347 (soccer.get_last_match): Liverpool F.C., include_stats true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 783 OUT329

✓

SimpleSportsRankingSeasonStrBFCL/simple_python_321 (sports_ranking): Barcelona, La Liga, 2021.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 791 OUT357

Tool Calling - Simple: Sports & Games (set 6) 4 passed

100.0%

IN20 226 OUT1 203

✓

SimpleBlackjackWinnerBFCL/simple_python_334 (blackjack.check_winner): player [A,10], dealer [10,9], ace_value 1.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 096 OUT360

✓

SimpleCreatePlayerProfileBFCL/simple_python_348 (create_player_profile): StarPlayer, class Mage, starting_level 5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 057 OUT171

✓

SimpleCurrentTeamRankingBFCL/simple_python_322 (sports_ranking.get_current): Liverpool, Premier League, season omitted or 2023-2024.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 023 OUT345

✓

SimplePlayerLastGameStatsBFCL/simple_python_304 (player_stats.getLastGame): LeBron James, Lakers, metrics {Points, Rebounds}.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 050 OUT327

Tool Calling - Simple: Stats, Finance & ML (set 1) 8 passed

100.0%

IN51 703 OUT4 580

✓

SimpleCardProbabilityBFCL/simple_python_112 (calculate_probability): total_outcomes=52, favorable_outcomes=4 (king from deck), round_to omitted or 2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 411 OUT270

✓

SimpleCompoundInterestNBFCL/simple_python_131 (calculate_compound_interest): principal=10000, rate=0.05, time=10, n=4.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 509 OUT966

✓

SimpleInvestmentProfitBFCL/simple_python_134 (investment.predictProfit): investment_amount=5000, annual_return=0.07, years=5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 454 OUT552

✓

SimpleLightTravelTimeBFCL/simple_python_100 (light_travel_time): distance_in_light_years=4, speed_of_light omitted or default.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 417 OUT208

✓

SimpleQuarterlyDividendBFCL/simple_python_128 (finance.calculate_quarterly_dividend_per_share): total_payout=50000000, outstanding_shares=100000000.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 465 OUT336

✓

SimpleRandomForestTrainBFCL/simple_python_109 (random_forest.train): n_estimators=100, max_depth=5, data='my_data'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 443 OUT277

✓

SimpleStockPriceByDateBFCL/simple_python_142 (get_stock_price): company_name Amazon/AMZN, date=2022-03-11, exchange NASDAQ or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 445 OUT959

✓

SimpleTTestStatsBFCL/simple_python_118 (stats.t_test): array_1=[10,15,12,14,11], array_2=[18,16,17,20,22], alpha=0.05.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 559 OUT1 012

Tool Calling - Simple: Stats, Finance & ML (set 2) 8 passed

100.0%

IN53 920 OUT5 819

✓

SimpleCalculateSpeedBFCL/simple_python_101 (calculate_speed): distance=450, time=20, to_unit=km/h.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 717 OUT279

✓

SimpleCardProbabilitySingleBFCL/simple_python_116 (probabilities.calculate_single): total_outcomes=52, event_outcomes=4 (king), round omitted or 2.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 705 OUT336

✓

SimpleCompoundInterestFreqBFCL/simple_python_136 (compound_interest): principal=10000, annual_rate=5.0, compounding_freq=monthly, time_in_years=5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 742 OUT1 195

✓

SimpleDiscountedCashFlowBFCL/simple_python_129 (calculate_discounted_cash_flow): coupon_payment=100, period=5, discount_rate=0.04, face_value omitted or 1000.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 737 OUT1 020

✓

SimpleHousePriceBedroomsBFCL/simple_python_110 (predict_house_price): bedrooms=3, bathrooms=2, area=1800, location San Francisco.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 738 OUT287

✓

SimpleStockPriceDaysBFCL/simple_python_143 (get_stock_price): company=AAPL, days=5, exchange NASDAQ or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 698 OUT552

✓

SimpleStockReturnProjectedBFCL/simple_python_137 (calculate_stock_return): investment_amount=5000, annual_growth_rate=0.06, holding_period=5, dividends omitted or false.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 750 OUT886

✓

SimpleTTestIndBFCL/simple_python_119 (hypothesis_testing.ttest_ind): sample1=[22,33,42,12,34], sample2=[23,45,44,14,38], significance_level=0.05.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 833 OUT1 264

Tool Calling - Simple: Stats, Finance & ML (set 3) 8 passed

100.0%

IN52 210 OUT5 218

✓

SimpleCompoundedInterestAnnuallyBFCL/simple_python_145 (calculate_compounded_interest): principal=5000, interest_rate=0.05, period=10, compounding_frequency omitted or Annually.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 523 OUT414

✓

SimpleDistanceEarthMoonBFCL/simple_python_102 (calculate_distance): body1=Earth, body2=Moon, unit mi/miles/mile.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 479 OUT316

✓

SimpleHeartRatioProbabilityBFCL/simple_python_117 (probability_of_event): success_outcomes=13 (hearts), total_outcomes=52, format_as_ratio=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 511 OUT399

✓

SimpleHousePriceRoomsBFCL/simple_python_125 (predict_house_price): area=2500, rooms=5, year=1990, location San Francisco/SF.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 538 OUT450

✓

SimpleMutualFundReturnBFCL/simple_python_139 (estimate_mutual_fund_return): yearly_yield=5.0, investment_amount=2000, years=3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 539 OUT448

✓

SimpleReturnOnEquityBFCL/simple_python_132 (calculate_return_on_equity): net_income=2000000, shareholder_equity=10000000, dividends_paid=200000.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 581 OUT1 966

✓

SimpleStockPriceCurrentMultiBFCL/simple_python_149 (get_stock_price): company_names=[Apple, Microsoft] (or tickers AAPL, MSFT).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 461 OUT291

✓

SimpleTwoSampleTTestRunBFCL/simple_python_120 (run_two_sample_ttest): group1=[3,4,5,6,4], group2=[7,8,9,8,7], equal_variance=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 578 OUT934

Tool Calling - Simple: Stats, Finance & ML (set 4) 8 passed

100.0%

IN52 876 OUT4 972

✓

SimpleBinomialProbDistBFCL/simple_python_114 (prob_dist.binomial): trials=10, successes=5, p omitted or 0.5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 574 OUT383

✓

SimpleFutureValueMonthlyBFCL/simple_python_133 (finance.predict_future_value): present_value=5000, annual_interest_rate=0.05, compounding_periods_per_year=12, time_years=3.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 616 OUT1 292

✓

SimpleGeometryAreaTriangleBFCL/simple_python_104 (geometry.area_triangle): base=6, height=10, unit omitted or 'square meters'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 557 OUT240

✓

SimpleMathPowerBFCL/simple_python_105 (math.power): base=3, exponent=4, mod omitted or 1.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 550 OUT193

✓

SimpleNPVCashFlowsBFCL/simple_python_127 (calculate_NPV): cash_flows=[200,300,400,500], discount_rate=0.1, initial_investment=2000.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 676 OUT891

✓

SimpleReturnOnInvestmentBFCL/simple_python_135 (calculate_return_on_investment): purchase_price=20, sale_price=25, dividend=2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 593 OUT450

✓

SimpleStockPriceDataTypeBFCL/simple_python_146 (stock_price): company Amazon/AMZN, days=3, data_type omitted or Close.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 551 OUT279

✓

SimpleTwoSampleTTestFloatBFCL/simple_python_123 (hypothesis_testing.two_sample_t_test): group1=[12.4,15.6,11.2,18.9], group2=[10.5,9.8,15.2,13.8], alpha omitted or 0.05.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 759 OUT1 244

Tool Calling - Simple: Stats, Finance & ML (set 5) 8 passed

100.0%

IN53 414 OUT7 361

✓

SimpleAreaUnderCurvePolyBFCL/simple_python_103 (mathematics.calculate_area_under_curve): polynomial=[3.0,2.0,-4.0], limits=[-1.0,2.0].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 665 OUT1 038

✓

SimpleBinomialCalcBFCL/simple_python_115 (calculate_binomial_probability): number_of_trials=8, number_of_successes=5, probability_of_success omitted or 0.5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 644 OUT361

✓

SimpleCalculateBmiBFCL/simple_python_107 (calculate_bmi): weight=70, height=175, system omitted or metric.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 643 OUT290

✓

SimpleFutureValueInvestmentBFCL/simple_python_148 (finance.calculate_future_value): initial_investment=20000, rate_of_return=0.08, years=5, contribution omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 683 OUT496

✓

SimpleLinearRegressionBFCL/simple_python_108 (run_linear_regression): predictors=[Age,Income,Education], target=Purchase_Amount, standardize=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 669 OUT245

✓

SimpleNPVInvestmentBFCL/simple_python_130 (finance_calculator.npv): cash_flows=[-50000,10000,15000,20000,25000,30000], discount_rate=0.08, years omitted or empty.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 767 OUT1 535

✓

SimpleStockPricesDurationBFCL/simple_python_147 (get_stock_prices): companies=[Microsoft,Google], duration='2 weeks'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 629 OUT293

✓

SimpleTTestDatasetsBFCL/simple_python_124 (t_test): dataset_A=[12,24,36], dataset_B=[15,30,45], alpha omitted or 0.05.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 714 OUT3 103

Tool Calling - Simple: Stats, Finance & ML (set 6) 10 passed

100.0%

IN71 477 OUT4 773

✓

SimpleBinomialCoin100BFCL/simple_python_121 (calc_binomial_prob): num_trials=100, num_success=60, prob_success=0.5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 153 OUT727

✓

SimpleCagrBFCL/simple_python_140 (calculate_cagr): initial_value=2000, final_value=3000, period_in_years=4.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 172 OUT381

✓

SimpleChiSquaredTestBFCL/simple_python_122 (chi_squared_test): table=[[10,20],[30,40]], alpha omitted or 0.05.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 162 OUT980

✓

SimpleDiceRollBFCL/simple_python_113 (probability.dice_roll): desired_number=6, number_of_rolls=2, die_sides omitted or 6.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 120 OUT342

✓

SimpleMarketPerformanceBFCL/simple_python_144 (market_performance.get_data): indexes=[S&P 500, Dow Jones], days=5, detailed omitted or true/false.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 137 OUT429

✓

SimpleMetalPriceBFCL/simple_python_141 (get_metal_price): metal=Gold, measure=ounce.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 096 OUT263

✓

SimpleNormalVariateBFCL/simple_python_111 (random.normalvariate): mu=0, sigma=1.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 117 OUT305

✓

SimplePortfolioFutureValueBFCL/simple_python_138 (portfolio_future_value): stock=X, invested_amount=5000, expected_annual_return=0.05, years=7.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 175 OUT633

✓

SimpleRandomForestClassifierBFCL/simple_python_106 (train_random_forest_classifier): dataset=your_dataset_name, max_depth=5, n_estimators=100.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 156 OUT192

✓

SimpleRSquaredBFCL/simple_python_126 (linear_regression.get_r_squared): dataset_path=C:/data/cars.csv, independent_variables=[engine_size,fuel_economy], dependent_variable=car_price.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 189 OUT521

Tool Calling - Tool Selection: Area (Triangle/Circle) 1 passed

100.0%

IN4 535 OUT338

✓

SelectTriangleAreaHeronToolBFCL/multiple_1 (math.triangle_area_heron): triangle area from three side lengths (Heron).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 535 OUT338

Tool Calling - Tool Selection: Art & Music 9 passed

100.0%

IN105 612 OUT3 635

✓

SelectAnalyzeStructureToolBFCL/multiple_162 (analyze_structure): dynamic structural analysis of specific floors.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN11 764 OUT333

✓

SelectConcertSearchToolBFCL/multiple_166 (concert.search): concert search by genre/location/date/price.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 730 OUT358

✓

SelectGuitarPriceToolBFCL/multiple_165 (guitar_price.find): guitar price by model/condition/location (not instrument_price.get).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN11 733 OUT586

✓

SelectInstrumentPriceToolBFCL/multiple_164 (instrument_price.get): instrument price by brand/model/finish (not guitar_price.find).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN11 738 OUT389

✓

SelectMelodyGeneratorToolBFCL/multiple_167 (music_generator.generate_melody): melody generation with key/note/length/tempo. Tightened per methodology: prompt explicitly gives "120 beats per minute", so tempo is required exactly 120 (ground truth also accepts omission, which equals the schema default).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 767 OUT261

✓

SelectMetropolitanArtworksToolBFCL/multiple_163 (metropolitan_museum.get_top_artworks): top artworks sorted by popularity.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 732 OUT380

✓

SelectMusicalScaleToolBFCL/multiple_169 (musical_scale): musical scale for a key.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 706 OUT526

✓

SelectSculptureInfoToolBFCL/multiple_160 (get_sculpture_info): sculpture info by artist with detail.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 717 OUT503

✓

SelectSongLyricsToolBFCL/multiple_168 (get_song_lyrics): lyrics by song title/artist.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 725 OUT299

Tool Calling - Tool Selection: Calculators & Directories 8 passed

100.0%

IN76 779 OUT3 200

✓

SelectCalculateGcdToolBFCL/multiple_35 (calculate_gcd): greatest common divisor, not lcm.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 591 OUT208

✓

SelectCurrencyGetRateToolBFCL/multiple_17 (currency_conversion.get_rate): rate at a specific date, not convert/latest-rate.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 610 OUT362

✓

SelectGeodistanceToolBFCL/multiple_65 (geodistance.find): distance between cities, not timezone difference/flights.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 580 OUT496

✓

SelectLibrarySearchBookToolBFCL/multiple_38 (library.search_book): search for a book, not reserve one.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 592 OUT402

✓

SelectLinearRegressionToolBFCL/multiple_53 (linear_regression): linear regression, not random forest.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 605 OUT452

✓

SelectMagneticFieldCalculateToolBFCL/multiple_41 (magnetic_field.calculate): magnetic field via Ampere's law, not electric field/force.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 623 OUT531

✓

SelectSolveQuadraticEquationToolBFCL/multiple_28 (solve.quadratic_equation): solve a quadratic, not rgb/string-reverse.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 599 OUT341

✓

SelectStockForecastToolBFCL/multiple_55 (stock_forecast): stock price prediction, not weather forecast.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 579 OUT408

Tool Calling - Tool Selection: Finance & Legal 8 passed2 failed

80.0%

IN138 223 OUT7 227

✓

SelectCagrCalculationToolBFCL/multiple_130 (calculate_cagr): compound annual growth rate from initial/final/years, not other finance tools.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 172 OUT396

✓

SelectCaseInfoToolBFCL/multiple_135 (get_case_info): retrieve case info (victim) by docket/court.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 148 OUT861

×

SelectCrimeRateToolBFCL/multiple_136 (get_crime_rate): crime rate by city/state/type/year.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.type.ToLower() == "violent"

Failed

IN13 140 OUT1 265

✓

SelectCrimeRecordToolBFCL/multiple_134 (crime_record.get_record): detailed crime record by case number/county.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 140 OUT456

✓

SelectFutureValueToolBFCL/multiple_132 (finance.calculate_future_value): future value from rate/initial/years, not compound_interest.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 175 OUT411

×

SelectLawsuitDetailsToolBFCL/multiple_139 (lawsuit_details.find): patent lawsuit details by company/year/type.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN19 881 OUT1 451

✓

SelectLawsuitSearchToolBFCL/multiple_137 (lawsuit_search): lawsuit search by company/date/location/status. Tightened per methodology: prompt explicitly asks for "ongoing" lawsuits, so status is required exactly "ongoing" (ground truth also accepts omission, but an explicitly-stated value must be passed).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 158 OUT641

✓

SelectLegalCaseFetchToolBFCL/multiple_138 (legal_case.fetch): fetch detailed legal case by id.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 112 OUT652

✓

SelectMarketPerformanceToolBFCL/multiple_131 (market_performance.get_data): index performance over N days.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 137 OUT714

✓

SelectMutualFundBalanceToolBFCL/multiple_133 (calculate_mutual_fund_balance): mutual fund balance from investment/yield/years.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 160 OUT380

Tool Calling - Tool Selection: Finance & Markets 8 passed

100.0%

IN82 450 OUT3 603

✓

SelectAvgClosingPriceToolBFCL/multiple_56 (avg_closing_price): average closing price, not volume/total revenue.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 275 OUT633

✓

SelectCompoundInterestCalcToolBFCL/multiple_49 (calc_Compound_Interest): compound interest, not future value/simple interest.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 324 OUT348

✓

SelectCurrencyConversionToolBFCL/multiple_52 (currency_conversion): currency conversion, not unit conversion.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 285 OUT483

✓

SelectDividendDataToolBFCL/multiple_54 (corporate_finance.dividend_data): dividend history, not generic stock market data.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 276 OUT359

✓

SelectHousePriceForecastToolBFCL/multiple_50 (house_price_forecast): house price forecast, not weather/stock forecast.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 269 OUT321

✓

SelectPropertyDepreciationToolBFCL/multiple_14 (finance.property_depreciation): depreciation, not loan/inflation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 324 OUT525

✓

SelectRevenueForecastToolBFCL/multiple_13 (corporate_finance.revenue_forecast): revenue forecast, not product price.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 316 OUT509

✓

SelectYoyGrowthRateToolBFCL/multiple_12 (corporate_finance.calculate_YOY_growth_rate): YOY growth, not ROE/ROA.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 381 OUT425

Tool Calling - Tool Selection: Food & Travel 7 passed

100.0%

IN64 582 OUT3 547

✓

SelectCurrencyConvertToolBFCL/multiple_95 (currency_exchange.convert): currency conversion, not unit_conversion. live_conversion is required true: prompt says "using current exchange rate" (ground truth = [true]).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 223 OUT464

✓

SelectDrivingDistanceToolBFCL/multiple_104 (get_shortest_driving_distance): driving distance, not instrument price / biology / scientist lookup.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 212 OUT453

✓

SelectHotelBookToolBFCL/multiple_93 (hotel.book): book a hotel room, not car.rental.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 247 OUT400

✓

SelectRecipeFindToolBFCL/multiple_91 (recipe.find): recipe by main ingredient + limit, not restaurant.find.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 211 OUT970

✓

SelectRecipeSearchToolBFCL/multiple_89 (recipe_search): find a recipe from ingredients, not restaurant_search/ingredient_replace.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 235 OUT474

✓

SelectRestaurantForGroupToolBFCL/multiple_90 (restaurant.find_group): restaurant for a group, not events.find_event.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 226 OUT336

✓

SelectWalmartVeganToolBFCL/multiple_92 (walmart.vegan_products): the named store is Walmart, not Safeway/Whole Foods.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 228 OUT450

Tool Calling - Tool Selection: Games, Food & Utilities 10 passed

100.0%

IN138 005 OUT4 668

✓

SelectBestGroceryStoreToolBFCL/multiple_188 (grocery_store.find_best): best grocery store by rating/products.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 832 OUT478

✓

SelectClosestRestaurantToolBFCL/multiple_185 (restaurant_search.find_closest): closest restaurant by cuisine/amenities.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 797 OUT323

✓

SelectFindRecipeToolBFCL/multiple_186 (find_recipe): recipe by restrictions/type/time.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 802 OUT914

✓

SelectGamePlayerStatsToolBFCL/multiple_180 (game_stats.fetch_player_statistics): player stats by game/username/platform.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 794 OUT213

✓

SelectMultiplayerGameFinderToolBFCL/multiple_182 (multiplayer_game_finder): multiplayer game by platform/rating.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 798 OUT237

✓

SelectRecipeCaloriesToolBFCL/multiple_183 (recipe_info.get_calories): calories of a recipe from a website.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 788 OUT379

✓

SelectRecipeSearchToolBFCL/multiple_184 (recipe_search): recipe by restriction/ingredients/servings.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 808 OUT726

✓

SelectSoccerLastMatchToolBFCL/multiple_181 (soccer.get_last_match): last match with stats for a club.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 793 OUT335

✓

SelectTimezoneConvertToolBFCL/multiple_189 (timezone.convert): convert a time between timezones.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 794 OUT752

✓

SelectWholeFoodsPriceToolBFCL/multiple_187 (whole_foods.check_price): price of items at a Whole Foods location.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 799 OUT311

Tool Calling - Tool Selection: Geometry, Finance & Services 7 passed

100.0%

IN58 982 OUT3 413

✓

SelectAirQualityForecastToolBFCL/multiple_63 (air_quality_forecast): air quality, not weather/news.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 409 OUT317

✓

SelectFinancialCompoundInterestToolBFCL/multiple_57 (financial.compound_interest): compound interest, not simple interest.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 472 OUT1 056

✓

SelectGeometryRectangleToolBFCL/multiple_31 (geometry_rectangle.calculate): rectangle area+perimeter, not square/circle.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 425 OUT240

✓

SelectLawyerFinderToolBFCL/multiple_59 (lawyer_finder): find a lawyer by specialization, not rent a car.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 411 OUT357

✓

SelectLibrarySearchBooksToolBFCL/multiple_68 (library.search_books): search a specific library, not Google Books/Open Library.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 411 OUT317

✓

SelectMapsShortestPathToolBFCL/multiple_27 (maps.shortest_path): shortest path with transport mode, not route-time estimation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 433 OUT797

✓

SelectTrafficEstimateToolBFCL/multiple_66 (traffic_estimate): traffic for a time period, not raw distance/weather.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 421 OUT329

Tool Calling - Tool Selection: History & Knowledge 9 passed1 failed

90.0%

IN136 034 OUT7 303

✓

SelectDiscovererToolBFCL/multiple_155 (get_discoverer): discoverer of a discovery, with detail.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 576 OUT613

×

SelectEarliestReferenceToolBFCL/multiple_157 (get_earliest_reference): earliest reference of a person from a source.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.source.ToLower() == "historical records"

Failed

IN13 588 OUT1 054

✓

SelectEventDateToolBFCL/multiple_153 (get_event_date): date of a historical event.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 573 OUT343

✓

SelectHistoricalContribToolBFCL/multiple_156 (historical_contrib.get_contrib): scientist's contribution on a date.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 608 OUT1 052

✓

SelectHistoryKeyEventsToolBFCL/multiple_152 (history.get_key_events): key war events by country/years.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 621 OUT1 178

✓

SelectMostFollowedToolBFCL/multiple_151 (social_media_analytics.most_followed): most-followed by topic/sub-topics.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 610 OUT873

✓

SelectNeuronalActivityToolBFCL/multiple_150 (calculate_neuronal_activity): neuronal activity from synaptic rate/weight/decay.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 632 OUT599

✓

SelectPaintNeededToolBFCL/multiple_159 (calculate_paint_needed): paint quantity for a wall.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 639 OUT380

✓

SelectPapalBiographyToolBFCL/multiple_158 (religious_history.get_papal_biography): papal biography with contributions.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 590 OUT895

✓

SelectUsPresidentToolBFCL/multiple_154 (US_president.in_year): president in a given year. Tightened per methodology: prompt explicitly asks for the "full name", so full_name is required true (ground truth also accepts omission, but an explicitly-requested option must be passed).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 597 OUT316

Tool Calling - Tool Selection: History, Demographics & Art 8 passed2 skipped

100.0%

IN85 985 OUT4 177

✓

SelectArtworkPriceToolBFCL/multiple_74 (art_auction.fetch_artwork_price): artwork price, not library book search. Tightened per methodology: prompt explicitly asks "on all auction platforms", so require platform == "all" (ground truth also accepts omission, but an explicitly-given value must pass).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 764 OUT542

✓

SelectArtworkSearchToolBFCL/multiple_77 (artwork_search.find): artwork search, not tourist_attraction/park search.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 747 OUT354

–

SelectCustomSculptureToolBFCL/multiple_76 (sculpture.create_custom): custom sculpture (material), not painting (color).0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Skipped: Ambiguous action-vs-question item (item flaw, not model behavior): the prompt mixes an order ('I want to order a custom bronze sculpture of a horse') with a question ('What material options are available?'), so a competent model may either call sculpture_create_custom or answer the question in text - not deterministic. Kept for traceability.

Skipped

IN0 OUT0

✓

SelectEuropeanMonarchsToolBFCL/multiple_70 (european_history.get_monarchs): monarch lookup, not events/culture.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 748 OUT477

✓

SelectExhibitionInfoToolBFCL/multiple_79 (exhibition_info): museum exhibitions, not restaurant_info.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 754 OUT559

–

SelectMuseumInfoToolBFCL/multiple_78 (museum_info): museum info with a feature list, not generic tourist_spot_info.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Ambiguous single-call item (item flaw, not model behavior): a dedicated exhibition_info tool coexists with museum_info, so the 'exhibitions' part can reasonably route either to museum_info's features or to a separate exhibition_info call; a competent model may make one or two calls - not deterministic. Kept for traceability.

Skipped

IN0 OUT0

✓

SelectPaintColorTrendsToolBFCL/multiple_75 (paint_color.trends): paint color trend, not weather/house-price trends.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 732 OUT406

✓

SelectPopulationByCategoryToolBFCL/multiple_71 (get_population): population by category, not bureau statistics/demographics.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 756 OUT1 164

✓

SelectReligionOriginToolBFCL/multiple_73 (religion.get_origin): founder/origin, not core beliefs.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 737 OUT323

✓

SelectStatePopulationToolBFCL/multiple_72 (us_history.population_by_state_year): state population, not GDP.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 747 OUT352

Tool Calling - Tool Selection: Isolated Collisions: Exhibition, Hotel & Currency 3 passed

100.0%

IN21 029 OUT2 624

✓

SelectBookLuxuryHotelToolBFCL/multiple_191 (book_hotel): luxury room with view, stay_duration variant.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 061 OUT552

✓

SelectCurrencyConversionToolBFCL/multiple_192 (currency_conversion.convert): convert an amount between currencies.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 977 OUT263

✓

SelectExhibitionToolBFCL/multiple_161 (find_exhibition): top-rated sculpture exhibition by location/art_form.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN6 991 OUT1 809

Tool Calling - Tool Selection: Knowledge & Media 9 passed1 failed

90.0%

IN123 757 OUT6 925

✓

SelectBattleDetailsToolBFCL/multiple_18 (european_history.battle_details): battle details, not war/leader info.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 747 OUT689

✓

SelectDatabaseModifyColumnsToolBFCL/multiple_10 (database.modify_columns): delete columns, not create a backup.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 796 OUT455

✓

SelectFiveFactorModelToolBFCL/multiple_69 (five_factor_model.analyse): Big Five personality, not MBTI.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 818 OUT577

✓

SelectGameRewardsToolBFCL/multiple_26 (game_rewards.get): rewards, not scores/missions list.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 744 OUT1 025

✓

SelectGenerateSoundWaveToolBFCL/multiple_21 (generate_sound_wave): generate a sound, not play a file.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 771 OUT993

✓

SelectMostPointsSingleGameToolBFCL/multiple_22 (sports_data.basketball.most_points_single_game): single-game record, not season/career.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 747 OUT407

✓

SelectPlayerStatsToolBFCL/multiple_23 (basketball.player_stats.get): player stats, not game/team stats.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 779 OUT348

✓

SelectReligionSchismsToolBFCL/multiple_19 (religion_history.get_schisms): schisms, not councils/reformations.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 739 OUT870

✓

SelectSculpturePriceToolBFCL/multiple_20 (sculpture_price.calculate): commission price, not sculptor info/availability.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 753 OUT289

×

SelectVideoGameStorePriceToolBFCL/multiple_25 (video_games.store_price): store price, not store currency/on-sale.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN17 863 OUT1 272

Tool Calling - Tool Selection: Legal, Environment & Services 10 passed

100.0%

IN121 618 OUT4 530

✓

SelectAirQualityToolBFCL/multiple_144 (air_quality): AQI by location/date.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 151 OUT367

✓

SelectEmissionsToolBFCL/multiple_145 (calculate_emissions): annual CO2 from distance/fuel/efficiency.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 187 OUT616

✓

SelectHumidityForecastToolBFCL/multiple_142 (weather.humidity_forecast): humidity forecast by location/days.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 138 OUT364

✓

SelectLawsuitCheckCaseToolBFCL/multiple_141 (lawsuit.check_case): verify a case's closed status by id.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 157 OUT558

✓

SelectLawsuitsSearchToolBFCL/multiple_140 (lawsuits_search): lawsuits by company/location/year.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 152 OUT349

✓

SelectMapDirectionsToolBFCL/multiple_147 (map_service.get_directions): directions avoiding features.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN12 153 OUT353

✓

SelectRestaurantFindNearbyToolBFCL/multiple_146 (restaurant.find_nearby): nearby restaurants by cuisine/distance.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 148 OUT336

✓

SelectSentimentAnalysisToolBFCL/multiple_149 (sentiment_analysis): sentiment of a review text.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 158 OUT254

✓

SelectSlopeGradientToolBFCL/multiple_143 (calculate_slope_gradient): slope gradient between two coordinates. Tightened per methodology: prompt explicitly asks for the gradient "in degree", so unit is required exactly "degree" (ground truth also accepts omission, which equals the schema default).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN12 247 OUT516

✓

SelectStockInfoToolBFCL/multiple_148 (get_stock_info): detailed stock information by company.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 127 OUT817

Tool Calling - Tool Selection: Local Services & Weather 9 passed

100.0%

IN86 772 OUT4 030

✓

SelectHumidityTemperatureForecastToolBFCL/multiple_60 (humidity_temperature_forecast): humidity/temperature, not events/movies.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 632 OUT292

✓

SelectLandscapeArchitectToolBFCL/multiple_61 (landscape_architect.find_specialty): landscape architect, not home renovation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 647 OUT469

✓

SelectLawyerSearchToolBFCL/multiple_58 (lawyer.search): find a lawyer, not a doctor.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 618 OUT240

✓

SelectLibraryFindNearbyToolBFCL/multiple_48 (library.find_nearby): find a library, not a store.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 643 OUT569

✓

SelectNatureParkToolBFCL/multiple_62 (nature_park.find_nearby): find a nature park, not a restaurant.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 653 OUT570

✓

SelectRideHailingToolBFCL/multiple_39 (ride_hailing.get_rides): find a ride, not a grocery delivery.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 641 OUT260

✓

SelectRoutePlannerToolBFCL/multiple_24 (route_planner.calculate_route): plan a route, not chess-club lookup.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 657 OUT1 116

✓

SelectTranslateToolBFCL/multiple_67 (translate): translate text, not sentiment/word-count.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 634 OUT280

✓

SelectUvIndexToolBFCL/multiple_64 (uv_index.get_future): UV index, not rainfall/snowfall.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 647 OUT234

Tool Calling - Tool Selection: Mathematics 8 passed

100.0%

IN75 269 OUT2 441

✓

SelectCalculateIntegralToolBFCL/multiple_33 (calculate_integral): definite integral, not derivative.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 413 OUT312

✓

SelectCosineSimilarityToolBFCL/multiple_47 (cosine_similarity.calculate): cosine similarity, not correlation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 432 OUT470

✓

SelectDiceRollProbabilityToolBFCL/multiple_51 (dice_roll_probability): dice sum probability, not coin/card.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 407 OUT380

✓

SelectFunctionsIntersectToolBFCL/multiple_29 (functions.intersect): intersection of two functions, not zeros of one.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 409 OUT364

✓

SelectMathLcmToolBFCL/multiple_34 (math.lcm): least common multiple, not gcd/sqrt.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 399 OUT160

✓

SelectQuadraticRootsToolBFCL/multiple_11 (math_roots.quadratic): quadratic roots from coefficients, not cubic/polynomial.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 411 OUT414

✓

SelectRectangleAreaToolBFCL/multiple_30 (rectangle.area): rectangle area, not circle/triangle area.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 397 OUT152

✓

SelectSortListToolBFCL/multiple_46 (sort_list): sort a list, not filter/sum.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 401 OUT189

Tool Calling - Tool Selection: Mathematics & Geometry 7 passed

100.0%

IN70 323 OUT1 980

✓

SelectCircleAreaToolBFCL/multiple_97 (geometry.area_circle): circle area, not plot_sine_wave.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 026 OUT221

✓

SelectCircumferenceToolBFCL/multiple_98 (geometry.circumference): circle circumference, not the reference/time/melody tools.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 017 OUT218

✓

SelectDatabaseQueryToolBFCL/multiple_119 (database.query): structured DB query with conditions, not the score/ROI/probability distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN10 108 OUT252

✓

SelectDerivativeToolBFCL/multiple_99 (calculus.derivative): derivative at a point, not get_personality_traits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 050 OUT347

✓

SelectHcfToolBFCL/multiple_100 (math.hcf): highest common factor, not the scale-progression distractor.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 036 OUT196

✓

SelectSolveQuadraticToolBFCL/multiple_96 (solve_quadratic_equation): solve quadratic, not stock/portfolio tools.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 049 OUT554

✓

SelectTriangleAreaToolBFCL/multiple_121 (geometry.area_triangle): triangle area, not map directions / invention lookup.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 037 OUT192

Tool Calling - Tool Selection: Mixed Domains 7 passed1 skipped

100.0%

IN68 830 OUT2 965

✓

SelectAverageGradeToolBFCL/multiple_9 (calculate_average): selects average, not standard_deviation/highest_grade.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 853 OUT299

✓

SelectCapacitanceToolBFCL/multiple_6 (capacitance_calculator.calculate): selects capacitance, not resistance/magnetic_field. Tightened per methodology: prompt explicitly states "dielectric constant K is 1.0", so K is required exactly 1.0 (ground truth also accepts omission, but an explicitly-given value must be passed).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 855 OUT564

✓

SelectCapitalToolBFCL/multiple_2 (country_info.capital): selects capital lookup, not largest_city/population.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 754 OUT204

✓

SelectDisplacementToolBFCL/multiple_4 (kinematics.calculate_displacement): selects displacement, not final_speed.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 847 OUT533

✓

SelectEuclideanDistanceToolBFCL/multiple_3 (EuclideanDistance.calculate): selects distance, not angleToXAxis.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 799 OUT374

✓

SelectRealEstateSearchToolBFCL/multiple_8 (realestate.find_properties): property search with budget, not property_valuation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 858 OUT410

✓

SelectWeatherByCoordinatesDateToolBFCL/multiple_5 (weather.get_by_coordinates_date): historical-by-coordinates, not by-city/forecast.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 864 OUT581

–

SelectWildlifeGrowthToolBFCL/multiple_7 (wildlife_population.assess_growth): population growth, not ecological_impact.0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Skipped: Ambiguous single-call item (item flaw, not model behavior): the prompt asks for BOTH population growth AND impact on woodland, and two distinct tools genuinely fit (wildlife_population_assess_growth + ecological_impact_analyze); a competent human would reasonably make two calls, so a single-call ground truth is unfair. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Tool Selection: Music, Sports & Games 8 passed1 failed

88.9%

IN90 211 OUT3 865

✓

SelectBoardGameRecommendToolBFCL/multiple_86 (BoardGameGeek.recommend): BoardGameGeek store, not AmazonGameStore.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 019 OUT536

✓

SelectConcertTicketToolBFCL/multiple_81 (concert.book_ticket): concert ticket, not festival ticket.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 048 OUT260

✓

SelectGameUpdateToolBFCL/multiple_87 (games.update.find): game update/patch, not reviews/price.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 030 OUT229

✓

SelectMusicGenerateToolBFCL/multiple_82 (music.generate): musical piece from key/tempo, not raw audio.generate.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 008 OUT207

×

SelectMusicShopToolBFCL/multiple_80 (music_shop.find_nearby): music shop with lessons, not gym.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.services.Count == 1 && args.services[0].ToLower() == "violin lessons" Values: args.services.Count = 1

Failed

IN10 019 OUT385

✓

SelectPlayerCountToolBFCL/multiple_88 (video_games.get_player_count): active player count, not sales.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 027 OUT551

✓

SelectPlayerGoalsToolBFCL/multiple_83 (player_stats.get_all_time_goals): per-player goals, not team/league top scorer.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 008 OUT436

✓

SelectSoccerScoresToolBFCL/multiple_85 (soccer_scores.get_scores): soccer scores, not basketball.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 024 OUT236

✓

SelectTopGoalScorersToolBFCL/multiple_84 (getTopGoalScorers): top goal scorers, not top assists.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 028 OUT1 025

Tool Calling - Tool Selection: Physical Sciences 10 passed

100.0%

IN113 074 OUT5 223

✓

SelectBiologicalEnergyToolBFCL/multiple_43 (biological.calc_energy): energy from moles of substance, not biomass/work.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 291 OUT411

✓

SelectBiotSavartLawToolBFCL/multiple_40 (electromagnetism.biot_savart_law): magnetic field via Biot-Savart, not Ampere's law.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 294 OUT706

✓

SelectConeVolumeToolBFCL/multiple_32 (geometry.calculate_cone_volume): cone volume, not cone mass.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 274 OUT398

✓

SelectFinalTemperatureToolBFCL/multiple_42 (calculate_final_temperature): mixing two gas samples, not mass calculation.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 335 OUT377

✓

SelectFinalVelocityToolBFCL/multiple_37 (kinematics.final_velocity): final velocity from initial velocity + time, not wave velocity/distance.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 284 OUT359

✓

SelectGeologyEraToolBFCL/multiple_45 (geology.get_era): geological era date, not historical event date.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 267 OUT443

✓

SelectPopulationGeneticsNeToolBFCL/multiple_16 (population_genetics.calculate_ne): effective population size, not range-shift/carrying-capacity.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 335 OUT1 005

✓

SelectSolarFarmPotentialToolBFCL/multiple_15 (solarFarm.potential): solar (not wind) energy potential from coordinates + panel area.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 390 OUT539

✓

SelectSpeedFromRestToolBFCL/multiple_36 (kinematics.calculate_speed_from_rest): speed from rest over a distance, not acceleration.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 319 OUT624

✓

SelectWeightInSpaceToolBFCL/multiple_44 (calculate.weight_in_space): weight on a planet, not unit/currency conversion.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 285 OUT361

Tool Calling - Tool Selection: Physics & Biology 13 passed

100.0%

IN195 044 OUT8 634

✓

SelectCellProteinsToolBFCL/multiple_109 (cellbio.get_proteins): proteins by cell compartment, not find_exhibition / mutation lookup.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN14 970 OUT921

✓

SelectDensityToolBFCL/multiple_107 (calculate_density): mass/volume density, not mix_paint_color.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN14 997 OUT248

✓

SelectDisplacementToolBFCL/multiple_102 (calculate_displacement): kinematic displacement, not poker/musical distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 015 OUT477

✓

SelectElectricFieldToolBFCL/multiple_106 (calculate_electric_field_strength): electric field, not paint/cooking/group distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN14 993 OUT350

✓

SelectEvolutionPredictionToolBFCL/multiple_114 (prediction.evolution): evolutionary-rate prediction, not modify_painting / calculate_probability.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 005 OUT400

✓

SelectFinalSpeedToolBFCL/multiple_103 (calculate_final_speed): free-fall final speed, not chess/court/event distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN14 985 OUT361

✓

SelectFitnessToolBFCL/multiple_113 (calculate_fitness): evolutionary fitness from trait values/contributions, not chess/walmart/lawyer distractors. Percentages map to decimal contributions [0.4, 0.6].1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN15 079 OUT311

✓

SelectForestGrowthToolBFCL/multiple_112 (forest_growth_forecast): forest growth forecast, not get_scientist_for_discovery. include_human_impact is required true: prompt says "including human impact" (ground truth = [true]).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN14 998 OUT410

✓

SelectGenotypeFrequencyToolBFCL/multiple_111 (calculate_genotype_frequency): genotype frequency, not factorial / find_flute.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 000 OUT384

✓

SelectHeatCapacityToolBFCL/multiple_108 (calc_heat_capacity): gas heat capacity, not discounted_cash_flow.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 016 OUT492

✓

SelectLightTravelTimeToolBFCL/multiple_120 (light_travel_time): light travel time, not grocery/hotel/exchange distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN14 983 OUT321

✓

SelectMagneticFieldToolBFCL/multiple_105 (calculate_magnetic_field): magnetic field of a loop, not concert/lawsuit distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN14 999 OUT544

✓

SelectMutationTypeToolBFCL/multiple_110 (mutation_type.find): SNP mutation type, not get_collectables_in_season.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN15 004 OUT3 415

Tool Calling - Tool Selection: Sports & Card Games 10 passed

100.0%

IN134 189 OUT6 904

✓

SelectChessTopPlayersToolBFCL/multiple_177 (board_game.chess.get_top_players): top chess players by location/rating.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 423 OUT415

✓

SelectDefenseRankingToolBFCL/multiple_173 (get_defense_ranking): NBA best-defence ranking by season.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 421 OUT787

✓

SelectFindAthleteToolBFCL/multiple_172 (sports_db.find_athlete): athlete profile by name/sport.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 406 OUT1 515

✓

SelectFindCardToolBFCL/multiple_178 (find_card_in_deck): find a card by rank/suit in default deck.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 417 OUT172

✓

SelectGameWinnerToolBFCL/multiple_171 (game_result.get_winner): winner of a game between two teams on a date.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 444 OUT802

✓

SelectPokerFullHouseToolBFCL/multiple_179 (poker_probability.full_house): probability of a full house (default deck/hand).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 407 OUT1 564

✓

SelectSoccerPlayerStatsToolBFCL/multiple_170 (soccer_stat.get_player_stats): player stats by name/season.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 434 OUT341

✓

SelectSportsRankingToolBFCL/multiple_174 (sports_ranking): team ranking in a league.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 404 OUT309

✓

SelectTeamScheduleToolBFCL/multiple_176 (sports_team.get_schedule): team schedule for N games in a league.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 428 OUT438

✓

SelectTopPlayerToolBFCL/multiple_175 (sports_ranking.get_top_player): top player by sport/gender.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 405 OUT561

Tool Calling - Tool Selection: Statistics, Finance & Health 5 passed1 failed

83.3%

IN56 326 OUT15 867

✓

SelectBmiMetersToolBFCL/multiple_117 (calculate_BMI): BMI in kg/m, not soccer / hotel_booking distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN9 367 OUT263

✓

SelectGcdToolBFCL/multiple_101 (math.gcd): greatest common divisor, not get_top_cases.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN9 344 OUT214

×

SelectHotelPricingToolBFCL/multiple_94 (hotel_room_pricing.get): hotel room pricing, not car/flight pricing.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.roomType.ToLower() == "suite with queen size bed"

Failed

IN9 375 OUT1 017

✓

SelectReturnOnEquityToolBFCL/multiple_128 (calculate_return_on_equity): ROE from income/equity/dividends, not library/lyrics/law distractors. dividends_paid is required 200000: prompt explicitly supplies "dividends paid of $200,000".1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 445 OUT367

✓

SelectSingleProbabilityToolBFCL/multiple_124 (probabilities.calculate_single): single-event probability, not math.power / fetch_DNA_sequence.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 353 OUT368

✓

SelectTwoSampleTtestToolBFCL/multiple_125 (run_two_sample_ttest): two-group t-test, not restaurant/personality distractors. equal_variance is required true: prompt says "assuming equal variance" (ground truth = [true, ""]).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 442 OUT13 638

Tool Calling - Tool Selection: Statistics, Finance & Services 8 passed

100.0%

IN91 259 OUT3 611

✓

SelectBmiMetricToolBFCL/multiple_116 (calculate_bmi): BMI in kg/cm with unit option, not celebrity_net_worth.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 395 OUT298

✓

SelectCalculateProbabilityToolBFCL/multiple_123 (calculate_probability): probability from favorable/total, not lawsuit/house-price distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 367 OUT237

✓

SelectCompoundInterestToolBFCL/multiple_129 (compound_interest): compound interest, not lawsuits_search.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 420 OUT674

✓

SelectFindRestaurantsToolBFCL/multiple_115 (find_restaurants): restaurant search with count + dietary, not instrument/stock/sports.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 406 OUT300

✓

SelectLinearRegressionToolBFCL/multiple_122 (run_linear_regression): regression with predictors, not recipe/itinerary/cooking distractors. standardize is required true: prompt says "Also apply standardization" (ground truth = [true]).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN11 417 OUT246

✓

SelectMoviesByActorToolBFCL/multiple_118 (imdb.find_movies_by_actor): movies by actor+year, not flight.book / lawsuits_search.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 389 OUT415

✓

SelectQuarterlyDividendToolBFCL/multiple_127 (finance.calculate_quarterly_dividend_per_share): dividend per share, not lyrics/paint/movie distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN11 421 OUT229

✓

SelectTTestToolBFCL/multiple_126 (t_test): two-dataset t-test, not event_finder / triangle-area distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 444 OUT1 212

Tool Calling - Tool Selection: Travel & Biology 7 passed1 failed

87.5%

IN97 928 OUT4 440

×

SelectBookHotelToolBFCL/multiple_190 (book_hotel): book a room by hotel/location/room_type/start/nights.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: args.room_type.ToLower() == "single"

Failed

IN12 282 OUT305

✓

SelectCellProteinsToolBFCL/multiple_196 (cellbio.get_proteins): proteins in a cell compartment.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 212 OUT824

✓

SelectDistanceDurationToolBFCL/multiple_193 (maps.get_distance_duration): distance/duration between two places.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 231 OUT442

✓

SelectForestGrowthToolBFCL/multiple_199 (forest_growth_forecast): forest growth forecast with human impact.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 234 OUT371

✓

SelectGenotypeFrequencyToolBFCL/multiple_198 (calculate_genotype_frequency): genotype frequency from allele frequency.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 242 OUT330

✓

SelectHeatCapacityToolBFCL/multiple_195 (calc_heat_capacity): heat capacity for a gas by temp/volume. Tightened per methodology: prompt explicitly states the gas is "air", so gas is required exactly "air" (ground truth also accepts omission, which equals the schema default).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 258 OUT739

✓

SelectMuseumHoursToolBFCL/multiple_194 (get_museum_hours): museum opening hours on a day.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 223 OUT509

✓

SelectMutationTypeToolBFCL/multiple_197 (mutation_type.find): gene mutation type by SNP id.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 246 OUT920

Tool Calling - Tool Selection: Triangle Properties 1 passed

100.0%

IN4 613 OUT813

✓

SelectTrianglePropertiesToolBFCL/multiple_0 (triangle_properties.get): retrieves triangle dimensions/properties from 3 sides.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 613 OUT813

Qwen 3.8 27B FP8 Think: medium | Benchmark #38

Tool Calling Benchmark based on BFCL

Results by difficulty

Results by category