Ai4Users - Qwen 3.6 35B FP8 Instruct

Benchmark items
Tool Calling - Irrelevance: Architecture & Artwork 7 passed2 failed	77.8%	IN416 262 OUT6 892
✓	NoToolForApartmentRentCostBFCL/irrelevance_158: apartment rental cost asked; only a new-construction cost estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 463 OUT43
×	NoToolForBestJazzSingerBFCL/irrelevance_164: best Jazz singer asked; only a top-instrument-players search (singers are not instrument players) is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2	Failed	IN10 575 OUT274
✓	NoToolForCathedralCeilingDesignBFCL/irrelevance_157: how to design a cathedral ceiling asked; only a lookup of facts about an existing building is available (retrieves data, does not advise on design) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 459 OUT513
✓	NoToolForHolyBookDefinitionBFCL/irrelevance_144: definition of a holy book asked; only a search-within-a-named-holy-book tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 457 OUT93
✓	NoToolForLectureTeacherBFCL/irrelevance_167: teacher of upcoming lectures asked; only an upcoming-concert details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 460 OUT34
✓	NoToolForMostFamousSculptureBFCL/irrelevance_153: most famous sculpture asked; only a sculpting-tools lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 465 OUT411
✓	NoToolForPianoMaintenanceTipsBFCL/irrelevance_166: piano maintenance tips asked; only an instrument rental-prices lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 460 OUT563
×	NoToolForScreamArtistBFCL/irrelevance_159: artist behind 'The Scream' asked. NOTE: the only tool (artwork_search) requires a museum_location that the user did not provide, so it cannot be correctly called; BFCL labels this irrelevant. Kept faithfully with ground truth = no tool call (flagged as borderline).1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.Completed Values: response.Completed = false	Failed	IN381 458 OUT4 838
✓	NoToolForSculptureRockTypeBFCL/irrelevance_152: which rock suits a garden sculpture asked; only a sculpture 3D-model creator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 465 OUT123
Tool Calling - Irrelevance: Buildings, Sculpture & Music 6 passed1 failed1 skipped	85.7%	IN26 437 OUT1 600
–	NoToolForCelloInstrumentTypeBFCL/irrelevance_165: instrument classification of a cello asked. NOTE: BFCL labels this irrelevant, but the only tool (get_instrument_info with detail=type) arguably answers it; flagged for review as a borderline BFCL item. Kept faithfully with ground truth = no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_instrument_info' with detail='type' answers 'what type of instrument is a cello' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForDeclarationSignersBFCL/irrelevance_137: who signed the declaration asked; only a historical-event DATE lookup is available (returns a date, not the signatories) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 284 OUT251
✓	NoToolForEiffelTowerHeightBFCL/irrelevance_156: Eiffel Tower height asked; only an architecture-plan generator (creates a new design, does not retrieve facts) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 288 OUT60
✓	NoToolForFamousDancersBFCL/irrelevance_162: famous dancers asked; only a museum-visual-artists lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 290 OUT244
✓	NoToolForReformationInitiatorBFCL/irrelevance_145: who initiated the Reformation asked; only a religious-event YEAR lookup is available (returns a year, not a person) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 283 OUT80
✓	NoToolForScientistDefinitionBFCL/irrelevance_143: definition of "scientist" asked; only a named-historical-figure info lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 280 OUT344
✓	NoToolForSellingGuitarBFCL/irrelevance_163: how to sell a guitar asked; only an instrument-tuning tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 285 OUT338
×	NoToolForSimilarBlueColorBFCL/irrelevance_148: a color similar to blue asked; only a complementary-color tool (opposite, not similar) is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN6 727 OUT283
Tool Calling - Irrelevance: Cell Biology & Genetics 8 passed2 skipped	100.0%	IN30 331 OUT2 848
✓	NoToolForClottingBloodCellsBFCL/irrelevance_32: blood cell type responsible for clotting asked; only a nucleus-count-based cell-type classifier is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 792 OUT62
✓	NoToolForCommonAncestorBFCL/irrelevance_41: common ancestor of two animals asked (phylogenetics knowledge); only a fibonacci-sequence calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 791 OUT245
✓	NoToolForEvolutionaryAlgorithmInGameAIBFCL/irrelevance_43: how to apply an Evolutionary Algorithm in game AI asked (advice); only a creature-evolution simulator is available (wrong context).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 792 OUT1 044
✓	NoToolForGeneVariantFrequencyBFCL/irrelevance_36: gene-variant frequency in a population asked; only a dominant-trait-from-alleles tool is available (the true-match variant-frequency tool is deliberately isolated in A4).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 797 OUT355
–	NoToolForIdentifyingGeneticSequenceBFCL/irrelevance_33: interpret a genetic code sequence asked; only a species-identification-from-sequence tool is available (returns a species, not what the sequence means).0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'identify_species' (identifies a species from a genetic sequence) plausibly answers the query about a DNA sequence, so a competent model may call it - BFCL-mislabeled/borderline-relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForIncreasingDeerPopulationBFCL/irrelevance_39: how-to advice on growing an animal population asked; only a population-growth CALCULATOR (needs numeric rates) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 791 OUT246
✓	NoToolForLionMatingProcessBFCL/irrelevance_35: mating process of a species asked; only a dominant/recessive genetic-traits lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 788 OUT286
✓	NoToolForMitochondriaCountBFCL/irrelevance_29: number/count of mitochondria asked; only a cell-part FUNCTION lookup is available (wrong query direction).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 792 OUT198
✓	NoToolForPandaEvolutionaryHistoryBFCL/irrelevance_42: evolutionary history asked (a narrative fact); only a biodiversity-index calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 788 OUT412
–	NoToolForWhaleGeneSequenceBFCL/irrelevance_44: the actual gene sequence of a species asked; only a gene-sequence GENERATOR (needs numeric mutation rate + duration) is available.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'gene_sequencer' (generate gene sequences for evolutionary changes) directly answers the query - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
Tool Calling - Irrelevance: Color, Geometry & Population 10 passed1 failed	90.9%	IN424 400 OUT5 399
✓	NoToolForColorInComputerVisionBFCL/irrelevance_131: color meaning in computer vision asked; only a color-in-personality-psychology analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 714 OUT116
✓	NoToolForDeclarationSigningDateBFCL/irrelevance_135: signing date asked; only date-arithmetic (add days) and age tools are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 712 OUT45
✓	NoToolForElectricityDiscovererBFCL/irrelevance_140: who discovered electricity asked; only an electric-current calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 708 OUT211
✓	NoToolForHexColorMixBFCL/irrelevance_146: mixing two hex colors asked; only a prophet-details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 724 OUT315
✓	NoToolForIndependenceSigningDateBFCL/irrelevance_138: signing date asked; only an age calculator (needs birthdate + current date) and a date-adder are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 712 OUT69
✓	NoToolForMostVisitedMarketBFCL/irrelevance_161: most visited market asked; only a museum visitation-statistics lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 714 OUT209
✓	NoToolForNbaFinalWinnerBFCL/irrelevance_133: NBA final winner asked; only a historical-battle details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 715 OUT50
✓	NoToolForPicassoPaintingYearBFCL/irrelevance_151: year of a Picasso painting asked; only a sculpture-dimensions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 713 OUT130
✓	NoToolForRelativityAuthorBFCL/irrelevance_142: who proposed special relativity asked; only a chemical-element properties lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 717 OUT15
×	NoToolForSculptureCreatorBFCL/irrelevance_155: sculpture creator asked; only a max-seating-capacity calculator is available -> no tool fits.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL Assertion expression evaluated to false: response.Completed Values: response.Completed = false	Failed	IN387 255 OUT4 095
✓	NoToolForWhichColorsToMixBFCL/irrelevance_150: which colors to mix for a shade asked; only an RGB-to-Pantone converter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 716 OUT144
Tool Calling - Irrelevance: Cooking, Currency & Markets 7 passed1 failed	87.5%	IN40 007 OUT1 599
×	NoToolForBakerySourdoughBFCL/irrelevance_223: a BAKERY selling sourdough asked; only a grocery-shop product finder available -> no tool fits.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 4	Failed	IN17 295 OUT432
✓	NoToolForChickenRecipeBFCL/irrelevance_211: the RECIPE to cook chicken asked; only a cook-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 243 OUT372
✓	NoToolForChocolateCakeIngredientsBFCL/irrelevance_220: cake INGREDIENTS asked; only a cooking-TEMPERATURE tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 246 OUT144
✓	NoToolForCurrencyToGbpBFCL/irrelevance_233: CURRENCY conversion asked; only a WEATHER tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 248 OUT23
✓	NoToolForHowToBoilAnEggBFCL/irrelevance_212: best WAY to boil an egg asked; only a boiling-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 245 OUT239
✓	NoToolForLasagnaRecipeBFCL/irrelevance_219: the RECIPE for lasagna asked; only a recipe-CALORIE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 243 OUT22
✓	NoToolForLegExercisesBFCL/irrelevance_221: leg EXERCISES asked; only a grocery-list generator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 243 OUT241
✓	NoToolForPetStoreBFCL/irrelevance_224: a PET store asked; only a GROCERY-store locator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 244 OUT126
Tool Calling - Irrelevance: Dates & Elements 9 passed1 failed	90.0%	IN47 876 OUT1 672
✓	NoToolForHydrogenPropertiesBFCL/irrelevance_141: chemical properties of Hydrogen asked; only a scientist-contributions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 628 OUT362
✓	NoToolForLargestPlanetBFCL/irrelevance_139: largest planet asked; only a star-info lookup (needs a star name) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 628 OUT135
✓	NoToolForMostImportantProphetBFCL/irrelevance_147: most important prophet asked; only a two-color mixer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 628 OUT121
×	NoToolForMuseumVisitFrequencyBFCL/irrelevance_160: museum member visit frequency asked; only a single most-frequent-visitor lookup is available -> no tool fits.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 3	Failed	IN15 219 OUT414
✓	NoToolForPantoneOfSkyBlueBFCL/irrelevance_149: Pantone code of sky blue asked; only a paint-proportions calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 630 OUT222
✓	NoToolForPersonalityTypeBFCL/irrelevance_130: user asks what personality type they are; only a big-five calculator (which needs the user's rated answers) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 626 OUT105
✓	NoToolForStadiumSeatingCapacityBFCL/irrelevance_154: stadium seating capacity asked; only a sculpture-creator lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 629 OUT86
✓	NoToolForVicePresidentBFCL/irrelevance_136: Vice President asked; only a US-president-by-year lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 628 OUT63
✓	NoToolForWaterlooCasualtiesBFCL/irrelevance_132: casualty count of a battle asked; only a historical-event date lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 630 OUT128
✓	NoToolForWorldCupWinnerBFCL/irrelevance_134: sports World Cup winner asked; only a historical-battle outcome predictor is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 630 OUT36
Tool Calling - Irrelevance: Electromagnetism, Thermo & Data 10 passed	100.0%	IN34 065 OUT1 865
✓	NoToolForElectronRestMassBFCL/irrelevance_23: mass of an electron asked (a knowledge fact); only magnetic-field / current tools are available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 399 OUT64
✓	NoToolForFlightDurationBFCL/irrelevance_20: flight duration asked; only a currency converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 402 OUT88
✓	NoToolForFreezingPointBFCL/irrelevance_25: freezing point asked; only boiling-point calculators are available (wrong phase transition).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 409 OUT285
✓	NoToolForGasAmountGeneratedBFCL/irrelevance_26: amount of gas generated asked; only a gas-PRESSURE calculator is available (wrong quantity).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 422 OUT700
✓	NoToolForGiniCoefficientBFCL/irrelevance_48: Gini coefficient asked; only a GDP-fetch tool is available (different economic indicator).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 402 OUT108
✓	NoToolForHeatingWaterBFCL/irrelevance_27: heat energy to warm water asked. NOTE: the original tool (calculate_heat) is a fair match for this query (only the well-known specific heat is missing), which would make "no call" ambiguous; per human-baseline fairness it is deliberately NOT shown here (kept as a distractor in A3), so no visible tool fits and "no call" is unambiguously correct.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 413 OUT272
✓	NoToolForHexagonSidesBFCL/irrelevance_28: number of sides of a hexagon asked (basic geometry knowledge); only a boiling-point tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 400 OUT8
✓	NoToolForSymbolicMagneticFieldBFCL/irrelevance_22: magnetic field asked with SYMBOLIC values (distance 'r', current 'I'); the two magnetic-field tools require concrete numeric distance + current, so there is nothing to pass -> no fit.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 413 OUT206
✓	NoToolForTimeDifferenceBFCL/irrelevance_49: time-zone difference asked; only a geo-coordinate lookup tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 402 OUT118
✓	NoToolForWorldSeriesWinnerBFCL/irrelevance_53: sports trivia asked; only a SQL database-query runner is available (no database/query provided).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 403 OUT16
Tool Calling - Irrelevance: Entertainment, Sports & Knowledge 9 passed	100.0%	IN31 428 OUT738
✓	NoToolForBasketballResultBFCL/irrelevance_84: basketball game result asked; only a stock-price retrieval tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 492 OUT39
✓	NoToolForBestSoccerPlayerBFCL/irrelevance_79: best current soccer player asked; only a stock-market prediction tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 490 OUT37
✓	NoToolForCaliforniaGovernorBFCL/irrelevance_87: current Governor of California asked; only a court-case filing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 488 OUT56
✓	NoToolForCrimeThrillerMovieListBFCL/irrelevance_88: best crime-thriller movies of 2020 asked; only a document forgery detector is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 497 OUT193
✓	NoToolForFifaWorldCupWinnerBFCL/irrelevance_80: FIFA World Cup 2010 winner asked; only a stock-ticker lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 493 OUT49
✓	NoToolForFrenchTranslationBFCL/irrelevance_61: French translation of a word asked; only an SVM-model tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 492 OUT14
✓	NoToolForHorrorMovieListBFCL/irrelevance_77: horror movie recommendations asked; only an investment future-value tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 491 OUT296
✓	NoToolForPresidentialElectionWinnerBFCL/irrelevance_85: 2020 presidential election winner asked; only a criminal-case detail lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 493 OUT27
✓	NoToolForSportsNewsBFCL/irrelevance_76: latest sports news asked; only a portfolio investment-trend analyzer is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 492 OUT27
Tool Calling - Irrelevance: Finance & Economics 7 passed1 failed	87.5%	IN33 742 OUT1 112
✓	NoToolForAverageHousePriceBFCL/irrelevance_69: market-wide average house price asked; only a regression predictor (needs a trained model + feature vector, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 348 OUT89
✓	NoToolForCompoundInterestBFCL/irrelevance_70: compound-interest computation asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 374 OUT264
✓	NoToolForGoldPriceBFCL/irrelevance_78: today's gold spot price asked; only a bond-price calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 350 OUT63
✓	NoToolForHorrorMovieCatalogBFCL/irrelevance_81: horror movie recommendations asked; only a historical stock-price fetcher is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 351 OUT297
×	NoToolForLossProjectionBFCL/irrelevance_73: a loss projection asked; only a revenue-prediction tool is available (revenue != loss).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2	Failed	IN10 253 OUT195
✓	NoToolForPoundToKilogramBFCL/irrelevance_75: a mass unit conversion asked; only a portfolio-returns calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 349 OUT19
✓	NoToolForProfitMarginBFCL/irrelevance_71: profit margin asked; only an ROI calculator is available (ROI != margin) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 371 OUT144
✓	NoToolForStockNewsBFCL/irrelevance_82: stock news asked; only a capital-gains calculator is available (gains != news) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 346 OUT41
Tool Calling - Irrelevance: Finance Near-Matches 9 passed	100.0%	IN30 939 OUT2 075
✓	NoToolForBitcoinPriceBFCL/irrelevance_56: current bitcoin price asked; only a generic database record-select tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 428 OUT45
✓	NoToolForBookstoreLookupBFCL/irrelevance_55: which online bookstore sells a title asked; only an add-to-cart tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 433 OUT55
✓	NoToolForBusinessRateOfReturnBFCL/irrelevance_74: rate of return asked; only a net-profit calculator is available (profit != rate of return).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 449 OUT584
✓	NoToolForCoinFlipProbabilityBFCL/irrelevance_66: probability of a coin landing heads asked; only a standard-deviation tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 433 OUT42
✓	NoToolForCurrentInterestRateBFCL/irrelevance_83: the current market interest rate asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 427 OUT77
✓	NoToolForExternalRateOfReturnBFCL/irrelevance_72: EXTERNAL rate of return asked; only an internal-rate-of-return (IRR) tool is available (ERR != IRR, even though cash flows are given) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 457 OUT353
✓	NoToolForFinalPriceWithoutBasePriceBFCL/irrelevance_60: final price after discount+tax asked, but the original price is never given, so the (domain-matching) calculateFinalPrice cannot be correctly called -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 447 OUT181
✓	NoToolForFrontendDevelopmentBFCL/irrelevance_62: how to build an e-commerce frontend asked; only a recommender-model builder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 433 OUT684
✓	NoToolForHighestGrossingMovieBFCL/irrelevance_54: highest-grossing movie of all time asked; only a movie search (requires a specific title + year, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 432 OUT54
Tool Calling - Irrelevance: Food, Time & Distance 3 passed1 failed1 skipped	75.0%	IN16 440 OUT662
✓	NoToolForCakeBakingMachineBFCL/irrelevance_218: a baking MACHINE/appliance asked; only a recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 708 OUT138
✓	NoToolForDrivingTravelTimeBFCL/irrelevance_215: driving TRAVEL TIME asked; only a restaurant-bill tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 712 OUT90
×	NoToolForLondonTimezoneNameBFCL/irrelevance_226: the NAME of London's timezone asked; only a local-TIME-by-timezone tool available (it needs the timezone as input and returns a time, not the timezone name) -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2	Failed	IN8 304 OUT275
–	NoToolForPizzaInBostonBFCL/irrelevance_213: a place for pizza asked; a cuisine+location restaurant_finder is available. REVIEW FLAG (fairness): restaurant_finder(cuisine=pizza/Italian, location=Boston) plausibly answers this, so a competent human might legitimately call it. BFCL labels it irrelevant; kept as such and marked Hard because the intended "no call" is debatable. No OTHER query here is a restaurant search, so the tool cannot wrongly satisfy the file's other cases.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'restaurant_finder' (cuisine+location) answers 'good place for pizza in Boston' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForTriangleHypotenuseBFCL/irrelevance_238: the HYPOTENUSE of a triangle asked; only a geo lat/long distance calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 716 OUT159
Tool Calling - Irrelevance: Geo, Travel & Sentiment 6 passed2 failed1 skipped	75.0%	IN119 089 OUT3 159
✓	NoToolForAverageTemperatureBmiBFCL/irrelevance_110: an average December temperature asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 398 OUT66
×	NoToolForCaseLawyerBFCL/irrelevance_95: the lawyer of a case asked; only a case-outcome retriever is available -> no tool fits.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 9	Failed	IN39 923 OUT796
✓	NoToolForCurrentTimeSunriseBFCL/irrelevance_102: the current time asked; only a sunrise/sunset time calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 397 OUT25
✓	NoToolForFamousQuoteBFCL/irrelevance_119: a famous quote asked; only a sentiment-analysis tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 395 OUT32
✓	NoToolForFastestRouteBFCL/irrelevance_114: the fastest route asked; only a closest-airport finder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 399 OUT182
–	NoToolForRestaurantsInLondonBFCL/irrelevance_113: restaurants in London asked; only a generic points-of-interest locator is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'find_pois' (locate points of interest by category) answers 'restaurants in London' (restaurants are POIs) - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForSportsBrainImpactBFCL/irrelevance_120: the neurological impact of sports asked; only a caffeine neurological-impact tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 398 OUT243
✓	NoToolForTrafficConditionBFCL/irrelevance_112: the current traffic condition asked; only an address geocoder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 397 OUT57
×	NoToolForTravelTimeByCarBFCL/irrelevance_115: travel time by car asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 12	Failed	IN58 782 OUT1 758
Tool Calling - Irrelevance: Legal & Case Lookup 9 passed1 failed	90.0%	IN83 304 OUT1 798
✓	NoToolForAppleCaloriesBFCL/irrelevance_100: calories in an apple asked; only a litigation-cost calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 550 OUT38
✓	NoToolForBrainWeightBFCL/irrelevance_122: average human brain weight asked; only a per-patient neural-activity tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 552 OUT93
✓	NoToolForCaseJudgementBFCL/irrelevance_93: a case judgement asked; only a law-firm impactful-cases lister is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 550 OUT33
✓	NoToolForChineseRestaurantBFCL/irrelevance_97: a Chinese-restaurant recommendation asked; only a lawsuit-filing tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 553 OUT59
✓	NoToolForMovieRecommendationBFCL/irrelevance_116: a movie recommendation asked; only a word-count tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 551 OUT184
×	NoToolForNbaChampionBFCL/irrelevance_118: the 1996 NBA champion asked; only an NBA playoff-brackets display tool is available -> no tool fits.1 AI query 11 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 11	Failed	IN51 330 OUT848
✓	NoToolForPaintingEiffelSettlementBFCL/irrelevance_98: how long to paint the Eiffel Tower asked; only a lawsuit-settlement estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 554 OUT85
✓	NoToolForSecurityGuardSearchBFCL/irrelevance_92: finding a security guard asked; only a lawyer-search tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 560 OUT209
✓	NoToolForSphereVolumeBFCL/irrelevance_104: sphere volume asked; only a circular-park area tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 554 OUT226
✓	NoToolForTrafficLawsBFCL/irrelevance_99: traffic laws asked; only a lawsuit-search tool is available (lawsuits, not laws) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 550 OUT23
Tool Calling - Irrelevance: Math & Science Distractors 8 passed	100.0%	IN25 252 OUT1 157
✓	NoToolForDefiniteIntegralBFCL/irrelevance_9: definite integral asked; only an income-tax calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 159 OUT232
✓	NoToolForIntegralOfCubeBFCL/irrelevance_8: definite integral asked; only a string-to-int converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 152 OUT216
✓	NoToolForLatestMoviesBFCL/irrelevance_15: latest movie releases asked; only a physics final-velocity tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 146 OUT29
✓	NoToolForPerpendicularSlopeBFCL/irrelevance_3: slope of a perpendicular line asked; only a critical-points finder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 164 OUT60
✓	NoToolForPrimeFactorsBFCL/irrelevance_13: prime factorization asked; only a compound-interest calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 149 OUT55
✓	NoToolForRectangleAreaBFCL/irrelevance_6: rectangle area asked; only a circle-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 160 OUT56
✓	NoToolForTriangleAreaBFCL/irrelevance_0: triangle area asked; only a BMI tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 160 OUT157
✓	NoToolForUpwardAccelerationBFCL/irrelevance_14: acceleration asked (it is just gravity); the tempting max-height and final-velocity tools compute neither the acceleration nor what is asked -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 162 OUT352
Tool Calling - Irrelevance: Math & Statistics 5 passed1 failed	83.3%	IN22 133 OUT2 346
✓	NoToolForBinomialMeanBFCL/irrelevance_67: the MEAN of a binomial experiment asked; only p-value tools are available (they return a p-value, not the mean) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 028 OUT286
✓	NoToolForCivilLawComponentsBFCL/irrelevance_90: the components/categories of Civil Law asked; only a noise-complaint filing tool is available (filing a complaint does not explain law) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 000 OUT320
✓	NoToolForCoinTossMaxHeadsBFCL/irrelevance_63: how many heads CAN be obtained from tossing 3 coins (a count/maximum, not a probability) asked; only a probability-of-an-event calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 004 OUT88
×	NoToolForNullHypothesisStdDevBFCL/irrelevance_68: the STANDARD DEVIATION of a null-hypothesis test asked; only p-value tools are available (they return a p-value, not a standard deviation) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN7 089 OUT1 239
✓	NoToolForPowerOfTwoBFCL/irrelevance_59: raising a number to a power asked; only a linear-equation solver is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 003 OUT107
✓	NoToolForQuadraticEquationBFCL/irrelevance_57: solving a quadratic equation asked; only a genetic-algorithm optimizer and a LINEAR equation solver are available (a quadratic is not linear) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 009 OUT306
Tool Calling - Irrelevance: Medical, Social & Personality 6 passed3 skipped	100.0%	IN20 464 OUT800
✓	NoToolForBigMacCaloriesBFCL/irrelevance_123: calories of a Big Mac asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 409 OUT56
✓	NoToolForEconomicHappinessEffectBFCL/irrelevance_126: the effect of economic status on happiness asked; only a happiness-index lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 411 OUT102
✓	NoToolForFriendMbtiBFCL/irrelevance_129: a friend's MBTI asked; only a personality-type evaluator (needs trait percentages) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 409 OUT52
–	NoToolForLatestTechTrendBFCL/irrelevance_124: the latest technology trend asked; only a trending-topics-by-category tool is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_social_trends' (trending topics by category) answers 'latest trend in technology' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForMajorAirportsBFCL/irrelevance_117: major US airports asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 412 OUT184
–	NoToolForMotorNeuronDiseaseBFCL/irrelevance_121: motor neuron disease info asked; only a generic disease-info retriever is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'medical_records_get_disease_info' directly answers 'information on motor neuron diseases' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForPopularBooksBFCL/irrelevance_125: popular books by an author asked; only a recent-tweets retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 411 OUT176
–	NoToolForTwitterMoodBFCL/irrelevance_127: overall twitter mood asked; only a twitter sentiment-analysis tool is available -> BFCL ground truth is no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'sentiment_analysis_twitter' directly answers 'general mood of twitter regarding the new iPhone' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForVegetableServingsBFCL/irrelevance_128: daily vegetable servings asked; only a personality-test scorer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 412 OUT230
Tool Calling - Irrelevance: Mixed Distractors 8 passed	100.0%	IN26 008 OUT960
✓	NoToolForAirQualityBFCL/irrelevance_40: current air quality asked; only a plant-biomass calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 252 OUT93
✓	NoToolForFavoriteRestaurantBFCL/irrelevance_31: a favorite restaurant recommendation asked; only a cell-division simulator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 248 OUT40
✓	NoToolForGlobalWarmingDocumentaryBFCL/irrelevance_38: a documentary (video) asked; only a scientific-research-paper retriever is available (wrong medium).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 249 OUT199
✓	NoToolForLasagnaIngredientsBFCL/irrelevance_47: recipe ingredients asked; only a flight-timetable tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 249 OUT237
✓	NoToolForLionDominantTraitBFCL/irrelevance_34: dominant genetic trait of a species asked; only a gene-variant-FREQUENCY lookup is available (different genetic output).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 251 OUT133
✓	NoToolForMultinucleatedCellNameBFCL/irrelevance_30: name of a cell type with multiple nuclei asked; only a blood-cell classifier (by shape/size) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 255 OUT115
✓	NoToolForPicnicSpotBFCL/irrelevance_37: a picnic spot asked; only a local-fauna information tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 248 OUT122
✓	NoToolForTranslationBFCL/irrelevance_52: natural-language translation asked; only a file-compression tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 256 OUT21
Tool Calling - Irrelevance: Motion, Electricity & Misc 10 passed	100.0%	IN35 428 OUT1 937
✓	NoToolForCarProjectileRangeBFCL/irrelevance_16: projectile-style "how far will a car travel" asked with symbolic v/t/theta; the range tool needs concrete numeric velocity + angle -> nothing to pass -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 553 OUT546
✓	NoToolForConeVolumeBFCL/irrelevance_19: cone volume asked; only an investment-return calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 547 OUT216
✓	NoToolForCurrentClockTimeBFCL/irrelevance_17: current clock time asked; only a travel-time (distance/speed) calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 536 OUT31
✓	NoToolForElectronMassBFCL/irrelevance_24: mass of an electron asked (a knowledge fact); only an electric-current tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 537 OUT25
✓	NoToolForFibonacciOf20BFCL/irrelevance_51: fibonacci number asked; only a cryptocurrency-price tool is available (the fibonacci calculator lives in a different file so it cannot satisfy this query).1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 538 OUT77
✓	NoToolForForceAngleFromMomentumBFCL/irrelevance_18: angle of a force from momentum asked; only a vector-angle-from-XY-components tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 543 OUT325
✓	NoToolForHorrorMovieListBFCL/irrelevance_50: movie recommendations asked; only a Celsius-to-Fahrenheit converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 543 OUT288
✓	NoToolForMagneticFieldNearWireBFCL/irrelevance_21: magnetic field near a current-carrying wire asked; only an EM-wave-amplitude tool is available (magnetic-field calculators are deliberately kept in a different file).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 552 OUT276
✓	NoToolForSineOf45DegreesBFCL/irrelevance_45: sine of an angle asked; only a polygon-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 538 OUT108
✓	NoToolForTeslaPriceBFCL/irrelevance_46: product price of a car asked; only a currency-exchange-rate tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 541 OUT45
Tool Calling - Irrelevance: Music Theory & Lookup 8 passed	100.0%	IN25 588 OUT1 118
✓	NoToolForAncientGreekLawmakersBFCL/irrelevance_171: who established laws in Ancient Greece asked (history); only a musical-piece composer search is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 196 OUT324
✓	NoToolForBasketballGameAtVenueBFCL/irrelevance_169: who is playing a basketball game at a venue asked; only concert-locating tools exist.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 199 OUT105
✓	NoToolForBluesScalePatternBFCL/irrelevance_176: the interval pattern of a blues scale asked; only composer lookups are available (get_blues_scale lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 201 OUT168
✓	NoToolForDonQuixoteAuthorBFCL/irrelevance_172: author of the novel Don Quixote asked; only music-composition lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 194 OUT13
✓	NoToolForFamousUsComposersBFCL/irrelevance_170: most famous US composers asked (general knowledge, no specific piece); only a chord-progression creator + piece-specific composer lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 197 OUT231
✓	NoToolForGrammyBestAlbumBFCL/irrelevance_177: Grammy Award winner asked; only a song chord-progression lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 202 OUT72
✓	NoToolForPrimaryTriadsOfKeyBFCL/irrelevance_173: primary triads of a key asked; only a genre-common-chords finder is available (primary_triads lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 200 OUT121
✓	NoToolForUniversityClassAvailabilityBFCL/irrelevance_168: availability of a university class asked; only concert-availability / concert-search tools are available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 199 OUT84
Tool Calling - Irrelevance: Probability & Legal 5 passed	100.0%	IN14 007 OUT911
✓	NoToolForFaceCardProbabilityBFCL/irrelevance_64: probability of drawing a face card from a deck asked; only a COIN-toss heads probability tool is available (coins, not cards) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 798 OUT164
✓	NoToolForHomeElectricityNeedBFCL/irrelevance_58: home electricity demand (kWh) asked; only a solar-panel COUNT calculator is available (it sizes panels, not electricity need, and needs sunlight hours not given) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN2 801 OUT196
✓	NoToolForNoiseComplaintProcessBFCL/irrelevance_91: how/whether to report a noise complaint asked; only a law-category listing tool is available (it lists legal categories, it does not file or advise on complaints) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 798 OUT96
✓	NoToolForRedMarbleCountBFCL/irrelevance_65: number of red marbles asked; only a determine-population tool is available (it returns a total population from probability + sample size, not a sub-category count) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN2 813 OUT160
✓	NoToolForTenantRightsBFCL/irrelevance_89: explaining tenant rights asked; only a contract-generation tool is available (generating a contract does not explain rights) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN2 797 OUT295
Tool Calling - Irrelevance: Recipes & Cooking 6 passed1 failed1 broken1 skipped	75.0%	IN46 568 OUT63 165
×	NoToolForBestGameCharacterBFCL/irrelevance_206: best game character vs a dragon asked; only a best-WEAPON finder available -> no tool fits.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 5	Failed	IN22 371 OUT535
✓	NoToolForBestMovie2020BFCL/irrelevance_235: best MOVIE of 2020 asked; only a currency-exchange calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 458 OUT124
!	NoToolForBuildingAPcBFCL/irrelevance_209: how to build a PC asked; only a cooking-recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL Assertion expression evaluated to false: response.Completed Values: response.Completed = false	Broken	IN3 453 OUT62 144
✓	NoToolForFamousPlaceInParisBFCL/irrelevance_210: most famous PLACE in Paris asked; only a restaurant-by-dish search available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 455 OUT84
✓	NoToolForGameShortestPathBFCL/irrelevance_208: shortest in-game path asked; only a taxi-fare calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 463 OUT73
✓	NoToolForInchToMeterBFCL/irrelevance_234: LENGTH conversion (inches to metres) asked; only a CURRENCY converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 454 OUT15
✓	NoToolForLowestGameScoreBFCL/irrelevance_207: LOWEST game score asked; only a HIGHEST-score retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 458 OUT98
✓	NoToolForNewYorkPopulationBFCL/irrelevance_225: current POPULATION of New York asked; only a time-zone converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 456 OUT92
–	NoToolForQuickestFlightLondonTokyoBFCL/irrelevance_236: quickest way to fly London->Tokyo asked; a flight-duration tool is available. REVIEW FLAG (fairness): get_flight_duration plausibly answers this ("quickest ... by plane" ~= quickest flight duration), so a competent human might legitimately call it. Kept as BFCL labels it (irrelevant), but this item is borderline. Marked Hard because the intended "no call" is debatable.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_flight_duration' (with flight_type non-stop) plausibly answers 'quickest way ... by plane' - BFCL-mislabeled/borderline-relevant. Kept for traceability.	Skipped	IN0 OUT0
Tool Calling - Irrelevance: Restaurants & Travel 6 passed2 failed	75.0%	IN48 106 OUT897
×	NoToolForBeijingSunsetBFCL/irrelevance_227: SUNSET time asked; only a SUNRISE calculator available -> no tool fits.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 3	Failed	IN14 371 OUT329
✓	NoToolForClosestItalianRestaurantBFCL/irrelevance_216: FIND the closest Italian restaurant asked; only a currency converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 314 OUT50
✓	NoToolForFindSushiRestaurantBFCL/irrelevance_214: FIND a sushi restaurant asked; only a tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 316 OUT29
✓	NoToolForNearestPharmacyBFCL/irrelevance_237: FIND the nearest pharmacy asked; only a route-to-a-known-location planner available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 316 OUT81
✓	NoToolForPetFriendlyTrainStationBFCL/irrelevance_230: a pet-friendly TRAIN STATION asked; only hotel search tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 315 OUT52
✓	NoToolForSushiRestaurantsTokyoBFCL/irrelevance_229: FIND sushi RESTAURANTS asked; only hotel-booking tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 316 OUT34
✓	NoToolForThaiRestaurantChicagoBFCL/irrelevance_231: FIND a Thai RESTAURANT asked; only a hotel-availability checker available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 317 OUT23
×	NoToolForYogaRetreatBFCL/irrelevance_232: a yoga RETREAT asked; only a HOTEL search tool available. REVIEW FLAG (fairness): a retreat is not a hotel, but hotel_search_find_hotels accepts arbitrary amenities (spa, vegetarian) and a location, so it is somewhat tempting. BFCL labels this irrelevant (retreat != hotel); kept as such and marked Hard because the boundary is borderline.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 3	Failed	IN13 841 OUT299
Tool Calling - Irrelevance: Sports & Games (set 1) 9 passed1 failed	90.0%	IN46 937 OUT2 057
×	NoToolForAssistAverageBFCL/irrelevance_185: assist average asked; only an average-SCORING tool is available -> no tool fits.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 3	Failed	IN14 801 OUT300
✓	NoToolForBestChessMoveBFCL/irrelevance_194: best next move asked; only a board-game-STATISTICS tool is available (the move-suggesting board_analyser lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 574 OUT134
✓	NoToolForChessArrangementCountBFCL/irrelevance_196: total theoretical arrangements asked; only a final-score calculator is available (possible_moves lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 574 OUT239
✓	NoToolForFootballTeamRankingBFCL/irrelevance_186: ranking of a team asked; only a season-MVP tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 570 OUT148
✓	NoToolForIceCreamStoreBFCL/irrelevance_201: find a physical store asked; only a video-game guide is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 567 OUT130
✓	NoToolForMostHeartsInPokerBFCL/irrelevance_199: who holds most of a suit asked; only a poker-dealing tool is available (get_highest_card_holder lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 574 OUT188
✓	NoToolForPlayerBiographyBFCL/irrelevance_189: a biographical "who is X" asked; only a top-ranked-athlete lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 567 OUT80
✓	NoToolForTeamRosterBFCL/irrelevance_192: full roster asked; only a top-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 569 OUT77
✓	NoToolForUnoRulesBFCL/irrelevance_198: the rules of a game asked; only a deck-shuffling tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 571 OUT691
✓	NoToolForWorldCupWinnerBFCL/irrelevance_191: who won a tournament asked (no team named); only a per-team match-stats tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 570 OUT70
Tool Calling - Irrelevance: Sports & Games (set 2) 8 passed1 failed	88.9%	IN38 327 OUT957
✓	NoToolForBlackjackAceRuleBFCL/irrelevance_200: a rules question asked; only a poker highest-card-holder tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 469 OUT129
✓	NoToolForBookAuthorBFCL/irrelevance_205: author of a book asked; only a video-game score calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 473 OUT19
✓	NoToolForChessTournamentWinnerBFCL/irrelevance_195: who won a tournament asked; only a board-position move-suggester is available (the game-history fetch_game_stats lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 470 OUT123
✓	NoToolForCreateJigsawBFCL/irrelevance_204: CREATE a puzzle asked; only a puzzle-SOLVING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 463 OUT187
×	NoToolForHaloPlayerRankBFCL/irrelevance_203: a player's RANK asked; only a player-SCORE lookup is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2	Failed	IN10 584 OUT181
✓	NoToolForLakersTopScorerBFCL/irrelevance_193: top scorer of a team asked; only a team-details tool (roster/results/upcoming, not scorers) is available (sports_team.get_top_scorer lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 468 OUT125
✓	NoToolForMonopolyWinnerBFCL/irrelevance_197: who won a casual game asked; only a possible-moves calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 468 OUT32
✓	NoToolForTopTennisPlayerBFCL/irrelevance_190: top ranked PLAYER asked; only a TEAM standing/ranking tool is available (the athlete-ranking get_top_ranked lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 467 OUT123
✓	NoToolForWorldSeriesWinnerBFCL/irrelevance_202: who won a real game asked; only a score calculator (from raw points) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 465 OUT38
Tool Calling - Irrelevance: Sports Statistics 8 passed1 failed	88.9%	IN42 975 OUT1 031
✓	NoToolForBasketballSeasonMvpBFCL/irrelevance_187: who won the MVP asked; only a team-ranking tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 511 OUT66
✓	NoToolForClippersRosterBFCL/irrelevance_179: roster of a team asked; only a two-team game-results tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 504 OUT71
✓	NoToolForCommonChordsInPopBFCL/irrelevance_174: most common chords in a pop song asked; only a key-signature primary-triads tool is available (find_common_chords lives in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 509 OUT247
×	NoToolForCricketMatchParticipantsBFCL/irrelevance_180: who is playing in cricket matches today asked; only a cricket-schedule tool (dates/matches, not participants) is available -> no tool fits.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 3	Failed	IN14 905 OUT401
✓	NoToolForLaLigaPlayersBFCL/irrelevance_181: who played in a whole league asked; only a two-named-team last-match-result tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 504 OUT68
✓	NoToolForMoonlightSonataComposerBFCL/irrelevance_175: composer of Moonlight Sonata asked; only a blues-scale generator is available here (all composer-lookup tools are isolated in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 507 OUT39
✓	NoToolForNbaMvpWinnerBFCL/irrelevance_184: who won an MVP award asked (no player named); only a per-player stats tool that requires a player name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 513 OUT58
✓	NoToolForTopAssistPlayerBFCL/irrelevance_178: top ASSIST player asked; only a top-GOAL-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 508 OUT50
✓	NoToolForWimbledonChampionBFCL/irrelevance_183: winner of a tournament asked (no person named); only a celebrity-info tool that requires a name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 514 OUT31
Tool Calling - Irrelevance: Time, Weather & Misc 2 passed1 failed1 skipped	66.7%	IN10 350 OUT467
×	NoToolForCityDistanceKmBFCL/irrelevance_239: DISTANCE (km) between two cities asked; only a get_date tool that returns the TIME difference between two locations is available (its 'unit' arg mentions distance, but it computes time) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1	Failed	IN5 192 OUT223
–	NoToolForCurrentTimeSydneyBFCL/irrelevance_228: current time in Sydney asked; a local-time-by-city tool is available. REVIEW FLAG (fairness): get_local_time(location='Sydney') directly answers this, so a competent human would legitimately call it. BFCL nonetheless files it under irrelevance; kept as such to preserve the source label, but this item is effectively RELEVANT and should be reviewed. Marked Hard because the "no call" expectation is contradicted by the tool itself.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_local_time' directly answers 'current time in Sydney' - BFCL-mislabeled relevant. Kept for traceability.	Skipped	IN0 OUT0
✓	NoToolForTomatoCaloriesBFCL/irrelevance_222: CALORIES in a tomato asked; only a grocery-item-details lookup available. REVIEW FLAG (fairness): item_details at 'detailed' level could arguably surface nutrition info, and it requires a store_location the user never gave; BFCL labels it irrelevant (it is a grocery-store item lookup, not a nutrition-facts service). Kept as such; marked Medium as borderline.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN2 580 OUT44
✓	NoToolForWriteABookBFCL/irrelevance_217: "can you write a book" asked; only a recipe-CREATION tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN2 578 OUT200
Tool Calling - Irrelevance: Weather, Environment & Services 8 passed2 failed	80.0%	IN130 155 OUT3 064
✓	NoToolForBestCo2ReductionBFCL/irrelevance_108: the best way to reduce CO2 asked; only a numeric emission-reduction estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 662 OUT413
✓	NoToolForBestMonthToVisitBFCL/irrelevance_101: best month to visit Hawaii asked; only an average-monthly-temperature tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 660 OUT126
×	NoToolForCactusNutrientsBFCL/irrelevance_109: a plant's nutrient needs asked; only a weekly watering-needs calculator is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2	Failed	IN11 202 OUT262
✓	NoToolForCurrentTimeWeatherBFCL/irrelevance_103: the current time asked; only a current-weather forecast tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 661 OUT40
✓	NoToolForGardenSpeciesCompositionBFCL/irrelevance_107: garden species composition asked; only a soil sand/silt/clay composition analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN3 664 OUT49
×	NoToolForGmoYogaMatBFCL/irrelevance_111: a GMO yoga mat asked; only an eco-friendly product locator is available (GMO is not eco-friendly) -> no tool fits.1 AI query 17 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 17	Failed	IN89 646 OUT1 622
✓	NoToolForImpactfulCasesBFCL/irrelevance_94: a firm's impactful cases asked; only a single-case detail lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 671 OUT79
✓	NoToolForPaintingEiffelCaseResultBFCL/irrelevance_96: how long to paint the Eiffel Tower asked; only a law-case result lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 663 OUT87
✓	NoToolForPollinationFlowersBFCL/irrelevance_105: top pollination flower species asked; only a route elevation-profile plotter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL	Passed	IN3 665 OUT254
✓	NoToolForTomatoFertilizerBFCL/irrelevance_106: best fertilizer for tomatoes asked; only a soil-type analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN3 661 OUT132
Tool Calling - Parallel Calls: Art, Music & Local Services 1 passed9 failed1 skipped	10.0%	IN231 818 OUT8 503
–	CalculateDrivingDistancesShortestRouteBFCL/parallel_38: two calc_distance calls — New York→Washington DC and Los Angeles→San Francisco. Tightened per methodology: the prompt explicitly enables "shortest route", so shortest_route == true is required (schema default is false).0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Skipped: Ambiguous scope (item flaw, not model behavior): the trailing 'with optional parameter shortest route enabled' can reasonably apply to only the second leg (its nearest clause) or to both; the BFCL ground truth requires it on both, but a competent human would not answer deterministically. Kept for traceability.	Skipped	IN0 OUT0
×	CheckInstrumentAvailabilityInTwoCitiesBFCL/parallel_42: two check_instrument_availability calls — Yamaha P125 in Berlin and Madrid.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN16 045 OUT571
×	FindMusicStoresInTwoCitiesBFCL/parallel_41: two find_music_instrument_store calls — San Francisco and Chicago; instruments {Yamaha acoustic guitar, Kawai piano} in any order and either casing.1 AI query 28 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 28 expectedCount = 2	Failed	IN87 495 OUT2 699
✓	FindRockAndJazzConcertsInTwoCitiesBFCL/parallel_43: four concert_finder calls — {San Francisco, New York} x {rock, jazz} over the next month. time_period may be omitted (defaults to 30 days ≈ "next month"). Isolated from concert_find_nearby (parallel_44) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN11 591 OUT435
×	GetExhibitionDetailsForTwoMuseumsBFCL/parallel_40: two museum_exhibition_detail calls — 'Wonder of Nature' at the Louvre and 'Age of Reptiles' at the British Museum. visitor_type {child, adult} (unordered). Tightened per methodology: the prompt explicitly asks for cost "for children and adult", so both visitor types are required (schema default is ['adult'] only).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN16 297 OUT826
×	GetMostPlayedPopAndRockSongsBFCL/parallel_45: two musicCharts_getMostPlayed calls — most played Pop and Rock song in Australia. Optional duration omitted (ground truth accepts the schema default of 0).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 2	Failed	IN11 425 OUT244
×	GetMuseumInfoForTwoCitiesBFCL/parallel_39: two museum_info_get_info calls — Washington D.C. and Paris; details {Opening hours, Adult tickets, Child tickets} in any order.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN20 880 OUT879
×	GetRgbAndHexForThreeColorsBFCL/parallel_37: three color_converter_get_color_info calls — navy, purple, maroon; each with {RGB, HEX} (unordered).1 AI query 15 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 15 expectedCount = 3	Failed	IN27 998 OUT1 086
×	GetTeamRankingsInTwoLeaguesBFCL/parallel_47: two get_team_ranking calls — Barcelona in UEFA Champions League and Manchester United in La Liga.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN11 573 OUT309
×	MixPaintColorsForTwoRequestsBFCL/parallel_36: two paint_color_mixture calls — watercolor magenta and acrylic navy.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN16 065 OUT743
×	WinningPercentageForTwoTeamsTwoSeasonsBFCL/parallel_46: four calculate_winning_percentage calls — {Lakers, Bulls} x {2018, 2020}.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN12 449 OUT711
Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 1) 3 passed10 failed	23.1%	IN847 331 OUT23 695
×	AverageFourNumberSetsBFCL/parallel_132: four calculate_average calls — one per list of numbers (element order preserved).1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 4	Failed	IN26 095 OUT1 257
×	CalculateBmiFourPeopleMetricBFCL/parallel_134: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt states "All measurements are in the metric system", so unit == "metric" is required (also the default).1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 4	Failed	IN18 959 OUT950
×	CalculateCityDistancesInKilometersBFCL/parallel_135: three geo_distance_calculate calls — NY->LA->Miami->NY, in kilometers. Tightened: the tool defaults to miles but the prompt asks "in kilometers", so units == "kilometers" is REQUIRED (omission would return miles, contradicting the request — the ground truth's "omit" option is unsafe here).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 3	Failed	IN23 367 OUT613
✓	CalculateGpsDistancesAlongRouteBFCL/parallel_133: three calculate_distance calls over GPS coordinate pairs (Paris->Rome->Athens-> Giza), each in kilometers. Uses the coordinate-based tool (coord1/coord2), not the city-name distance tools that are also visible.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 905 OUT3 320
✓	CompareAverageTemperatureNyAndLaBFCL/parallel_128: two average_temperature calls — New York 7 days in Fahrenheit, Los Angeles 7 days in Celsius. Tightened: the prompt explicitly requests Fahrenheit for NY, so temp_unit == "Fahrenheit" is required (also the default); Celsius is non-default and required for LA.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 362 OUT192
✓	CreateTwoHistogramsWithFiveBinsBFCL/parallel_129: two create_histogram calls — one per data set, 5 bins each (data order preserved).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 826 OUT467
×	EstimateElephantPopulationThreeScenariosBFCL/parallel_125: three elephant_population_estimate calls — current 500 at 2%, 1.5%, 2.5% over 10y.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3	Failed	IN25 116 OUT1 506
×	FindItalianRestaurantsInNyAndLaBFCL/parallel_130: two find_restaurants calls — New York and Los Angeles, Italian, 4 results each, dietary_requirements [vegan, gluten-free]. Distinct from restaurant_find_nearby (this tool carries food_type + number, which the prompt supplies, so it is the correct choice here).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN23 375 OUT1 310
×	FindRestaurantsByDietInThreeCitiesBFCL/parallel_127: three restaurant_find_nearby calls — New York, Los Angeles, Chicago, each with dietary_preference [Vegan, Gluten-free, Dairy-free]. List order follows the prompt (matches ground truth).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN13 511 OUT834
×	PredictEvolutionDarwinAndLamarckBFCL/parallel_126: two prediction_evolution calls — African Elephant over 5000y, Darwin then Lamarck. Tightened per methodology: the prompt explicitly says "using the Darwin model", so model == "Darwin" is required for the first call (ground truth also accepts omission via the 'Darwin' default).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN17 879 OUT883
×	ProjectPopulationGrowthJapanAndIndiaBFCL/parallel_124: four population_projections calls — Japan/10y and India/20y, each at the current (default) growth rate and at an explicit rate. "current growth rate" maps to the default (growth_rate omitted, accepts null or 0.01); the explicit 1.5%/2.1% rates are required.1 AI query 16 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 4	Failed	IN26 025 OUT1 428
×	RouteThreeLegsAvoidingTollsBFCL/parallel_131: three map_routing_fastest_route calls — SF->Palo Alto->San Jose->home, all avoiding tolls. Tightened: the prompt explicitly says "avoiding toll roads", so avoid_tolls == true is required for every leg (non-default value stated in the prompt).1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN18 843 OUT811
×	ShortestCityDistanceBusWithAndWithoutTransferBFCL/parallel_136: two city_distance_find_shortest calls — NY->LA by bus, first without transfer then allowing transfer. Tightened: the prompt explicitly says "using a bus", so transportation == "bus" is required for both (also the default). allow_transfer: false/omitted for the first, true for the second.1 AI query 100 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.Completed Values: response.Completed = false	Failed	IN628 068 OUT10 124
Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 2) 5 passed9 failed	35.7%	IN209 392 OUT10 239
✓	CalculateAreaOfThreeTrianglesBFCL/parallel_143: three calc_area_triangle calls — (10,5), (15,7), (20,10).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 962 OUT456
×	CalculateBinomialProbabilityBasketballBFCL/parallel_161: three calculate_binomial_probability calls — basketball shooter at 60%: (10,7), (15,10), (20,15). probability_of_success == 0.6 is required (prompt states 60%). ISOLATED here (away from prob_dist.binomial in ToolCalling-Parallel-D3): both tools compute binomial probability, and this prompt does not name a function, so having both visible would make the choice genuinely ambiguous. Kept apart so exactly one correct tool is available.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN14 210 OUT1 467
×	CalculateBmiInMetersForThreePeopleBFCL/parallel_138: three calculate_BMI calls (weight kg, height meters) — John, Sarah, Mike.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3	Failed	IN25 661 OUT1 006
×	CalculateCelestialDistancesInMilesBFCL/parallel_149: two calculate_distance calls between celestial bodies (Mars-Venus, Mars-Jupiter), in miles. Uses the body-name distance tool (body1/body2). Tightened: prompt asks "in miles", so unit == "miles" is required (tool default is kilometers, so omission would be wrong).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN18 625 OUT976
×	CalculateClockAnglesAtThreeTimesBFCL/parallel_145: three calculate_clock_angle calls — 3:15, 8:20, 11:50. Tightened: the prompt asks for results "rounded to 2 decimal places", so round_to == 2 is required (also the default).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN19 555 OUT1 219
✓	CalculateFactorialOfFourNumbersBFCL/parallel_144: four math_factorial calls — 5, 3, 4, 2.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 972 OUT225
×	CalculateLightTravelTimeThreeLegsBFCL/parallel_147: three light_travel_time calls — 4.22, 6.1, 5.88 light years. Tightened: the prompt explicitly states the speed of light (299792458 m/s), so speed_of_light == 299792458 is required (also default).1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 3	Failed	IN24 949 OUT1 133
×	CalculateSpeedForCarBicycleTrainBFCL/parallel_148: three calculate_speed calls — car 500m/25s in km/h, bicycle 1000m/200s in m/s, train 10000m/600s in km/h. Each output unit is explicitly named in the prompt, so to_unit is required.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN19 809 OUT903
×	FetchAllDataFieldsForTwoEmployeesBFCL/parallel_139: two employee_fetch_data calls — Tech Solutions, employees 12345 and 67890, each fetching all four data fields (order follows the prompt / ground truth).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 775 OUT443
×	FindDiCaprioDramaAndComedyMoviesBFCL/parallel_140: two imdb_find_movies_by_actor calls — Leonardo DiCaprio, Drama in 2010 and Comedy in 2012. Tightened: the prompt explicitly names each category, so category is required for both.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 559 OUT458
✓	GetImaxAndTwoDTheaterReleasesBFCL/parallel_141: two get_theater_movie_releases calls — New York IMAX over 7 days, Los Angeles 2D over 14 days. Tightened: the prompt explicitly names each format, so format is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 793 OUT372
×	PlotTwoSineWavesBFCL/parallel_146: two plot_sine_wave calls with all five parameters supplied by the prompt.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 790 OUT452
✓	SortFourListsAscendingAndDescendingBFCL/parallel_137: eight array_sort calls — four lists, each sorted ascending AND descending (list element order preserved as given).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN9 848 OUT871
✓	UpdateTwoCustomersNameAndEmailBFCL/parallel_142: two update_user_info calls — users 12345 and 67890, both set name "John" and email "example@.com". update_info is a nested object. Tightened: the prompt names the 'CustomerInfo' database, so database == "CustomerInfo" is required (also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 884 OUT258
Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 3) 3 passed8 failed	27.3%	IN178 424 OUT7 886
×	AreaUnderCurveForTwoPolynomialsBFCL/parallel_150: two mathematics_calculate_area_under_curve calls — polynomial [3,-2,1] over [-1,2] and [1,0,-1] over [0,3] (coefficient/limit order preserved).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 149 OUT454
×	BinomialProbabilitiesThreeGamesBFCL/parallel_160: three prob_dist_binomial calls — (20,5), (50,15), (100,30), each with p == 0.3 (prompt states a 0.3 win probability). The prompt explicitly names 'prob_dist.binomial'; the colliding calculate_binomial_probability tool is kept in a separate script (ToolCalling-Parallel-D2).1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3	Failed	IN23 311 OUT1 355
✓	CalculateBmiSystemFourPeopleBFCL/parallel_154: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt says "use the metric system for all calculations", so system == "metric" is required (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 132 OUT517
×	DiceRollProbabilitiesThreeScenariosBFCL/parallel_159: three probability_dice_roll calls — roll 4 three times and roll 2 twice on a six-sided die (die_sides default 6, accepts omission), then roll 7 twice on an eight-sided die (die_sides == 8 required, non-default).1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN17 637 OUT992
✓	GenerateNormalVariatesTwoConfigsBFCL/parallel_158: four random_normalvariate calls — two at (mu 5, sigma 2) and two at (mu 10, sigma 3). Duplicate calls: each configuration must appear exactly twice (multiplicity 2).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 068 OUT504
×	PredictPricesForFourHousesBFCL/parallel_157: four predict_house_price calls — New York (3/2/1500), Los Angeles (4/3/2000), Chicago (2/1/1200), Miami (3/2/1800).1 AI query 16 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 4	Failed	IN25 370 OUT1 473
✓	RaiseTwoNumbersToPowersBFCL/parallel_152: two math_power calls — 3^5 and 2^3, no modulus (mod omitted / null).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 688 OUT139
×	RunLinearRegressionWithAndWithoutStandardizationBFCL/parallel_155: two run_linear_regression calls — same predictors [Age, Income, Education] and target "Spending Score", first without standardization then with. standardize is explicitly requested (false then true), so it is required for both.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN16 881 OUT566
×	TrainRandomForestFourConfigsBFCL/parallel_156: four random_forest_train calls on "data_random_forest" — (100,10), (200,20), (300,30), (400,40). The prompt explicitly names this function, disambiguating it from train_random_forest_classifier visible in this script.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 4	Failed	IN13 038 OUT545
×	TrainRfClassifierTwoConfigsBFCL/parallel_153: two train_random_forest_classifier calls — dataset1 (depth 10, 100 trees) and dataset2 (depth 20, 200 trees). The prompt explicitly names this function, disambiguating it from the other random-forest training tool (random_forest_train) visible in this script.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN16 869 OUT504
×	TriangleAreasInDifferentUnitsBFCL/parallel_151: three geometry_area_triangle calls in different units — (15,20) square meters, (25,30) square feet, (35,40) square inches. First unit is the default (accepts omission / square meters); feet and inches are non-default and required.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN29 281 OUT837
Tool Calling - Parallel Calls: Finance, Stats & Legal 2 passed11 failed	15.4%	IN237 380 OUT9 315
✓	BuildTwoDecisionTreeClassifiersBFCL/parallel_12: two model_DecisionTreeClassifier calls — gini/depth 5 and entropy/depth 10, both random_state 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 444 OUT251
×	CalculateFutureValueForThreeTermsBFCL/parallel_18: three financials_calculate_future_value calls — years 1, 5, 10 (same 1000 @ 0.05).1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 3	Failed	IN35 899 OUT653
×	CalculatePresentValueForThreeTermsBFCL/parallel_14: three calculate_present_value calls — years 10, 20, 30 (same payment 1000, rate 0.05).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN18 955 OUT735
×	CapitalGainsTaxForTwoStatesBFCL/parallel_15: two calculate_capital_gains_tax calls — CA (15000/25000) and FL (20000/50000).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 2	Failed	IN13 689 OUT529
×	ComputePythagorasForTwoTrianglesBFCL/parallel_10: two math_pythagoras calls — sides (3,4) and (5,12).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 932 OUT438
×	ConfidenceIntervalForTwoSampleSizesBFCL/parallel_13: two confidence_interval_calculate calls — sample sizes 50 and 150. Tightened per methodology: the prompt explicitly requests "95% confidence interval", so confidence_level == 0.95 is required (schema default is 0.9, but the prompt states the value).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN23 565 OUT1 133
✓	CourtCaseStatusAndTrialDatesBFCL/parallel_22: four court_info_get_case_status calls — cases XY1234 (LA) and GH5678 (Orange County), each queried for status (details may be omitted via the 'status' default) and for trial_date.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 758 OUT1 109
×	GetStockPriceAndVolumeForThreeCompaniesBFCL/parallel_17: three get_stock_data calls — AAPL, GOOG/GOOGL, MSFT; each with data_points {price, volume} (unordered).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN13 397 OUT479
×	LoanEligibilityForTwoInstitutionsBFCL/parallel_20: two loan_eligibility_check calls — HSBC (500k/100k) and Wells Fargo (700k/120k).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN18 203 OUT642
×	MortgagePaymentForThreeLoanTermsBFCL/parallel_19: three calculate_mortgage_payment calls — loan terms 15, 20, 30 (400000 @ 0.04).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN19 059 OUT758
×	PredictHousePriceInTwoCitiesBFCL/parallel_11: two ml_predict_house_price calls — New York 3000 sq ft, Los Angeles 4000 sq ft.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN17 853 OUT531
×	ReturnOnInvestmentGainAndLossBFCL/parallel_16: two calculate_return_on_investment calls — gain 500 on 2000, loss -1000 on 5000.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2	Failed	IN28 687 OUT940
×	SearchMoneyLaunderingConvictionsBFCL/parallel_21: two law_crimes_search calls — money laundering in San Francisco 2019 and Texas 2018.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN17 939 OUT1 117
Tool Calling - Parallel Calls: Legal, History & Environment 1 passed11 failed1 broken1 skipped	7.7%	IN1 338 732 OUT87 959
✓	BookTwoConnectingFlightsBFCL/parallel_30: two book_flight calls — San Francisco→Tokyo (2022-05-03) and Tokyo→Sydney (2022-05-18).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN9 061 OUT197
×	CalculateAlimonyForTwoDurationsBFCL/parallel_23: two alimony_calculator_ca_calculate calls — durations 10 and 20 years (10000/3000 income).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN19 409 OUT1 176
×	CalculateWasteForFamilyAndBachelorBFCL/parallel_29: two waste_calculation_calculate calls — LA family (2 adults, 2 children) and New York bachelor (1 single). The population argument is a nested object.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN19 373 OUT992
×	FetchTwoHistoricalFactsBFCL/parallel_31: two history_fact_fetch calls — Treaty of Paris and Magna Carta. Optional depth/year omitted (ground truth accepts the schema defaults).1 AI query 26 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 26 expectedCount = 2	Failed	IN86 830 OUT1 782
×	FindClassicalConcertsWithParkingBFCL/parallel_44: two concert_find_nearby calls — classical concerts near Berlin and Paris next Friday. Tightened per methodology: the prompt explicitly asks for concerts "with available parking", so amenities == ['Parking'] is required (this also matches the schema default, but the prompt states it). Isolated from concert_finder (parallel_43) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2	Failed	IN30 826 OUT956
–	FindCourtCasesByNumberAndTypeBFCL/parallel_26: four court_case_find calls — the cross product of case numbers {67813, 71249} and types {Civil, Criminal}; case_number is a single-element array; Civil may be omitted via its default. FLAGGED as borderline (see report): the prompt "case numbers '67813','71249' for type 'Civil' and 'Criminal'" is ambiguous about the pairing; BFCL ground truth expects all four combinations, kept faithful.0 AI queries 0 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL Skipped: Ambiguous call decomposition (item flaw, not model behavior): 'case numbers 67813, 71249 for type Civil and Criminal' has no single deterministic reading - two calls (both numbers per type, case_number as a list) is as valid as the BFCL 2x2 = four single-number calls; a competent human would not answer deterministically. Kept for traceability.	Skipped	IN0 OUT0
×	FindNatureReservesNearTwoCitiesBFCL/parallel_27: two nature_reserve_find_nearby calls — Berkeley (10km, picnic tables + restrooms) and Tokyo (5km, playgrounds + biking trails). Amenity list order is unordered.1 AI query 99 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.Completed Values: response.Completed = false	Failed	IN650 478 OUT10 466
×	GetCaseDetailsWithHistoryAndLitigantsBFCL/parallel_24: two law_case_get_details calls — cases 28473 and 64725. Tightened per methodology: the prompt explicitly asks for "their history and details of litigants", so include_history == true and include_litigants == true are required (schema defaults are false).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 961 OUT452
!	GetEventsForTwoPresidenciesBFCL/parallel_32: two us_history_events_by_presidency calls — Abraham Lincoln and George Washington. Optional start_year/end_year omitted (ground truth accepts the schema defaults).1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.Completed Values: response.Completed = false	Broken	IN38 439 OUT63 303
×	GetPresidentsAndVicePresidentsByYearBFCL/parallel_33: four get_president_and_vp calls — president for 1980 and 2016, vice president for 1975 and 2011.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4	Failed	IN20 867 OUT1 105
×	GetReligionInfoForTwoEmpiresBFCL/parallel_35: two ancient_empires_get_religion_info calls — Persian Empire and Mauryan Empire. Tightened per methodology: the prompt explicitly asks for "religious history and influences", so include_influences == true is required (schema default is false).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 895 OUT682
×	GetTemperatureForTwoCitiesBFCL/parallel_28: two get_current_and_future_temperature calls — Seattle and Los Angeles, next 3 hours.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 835 OUT254
×	LookupLawsuitsForTwoYearsBFCL/parallel_25: two lawsuit_lookup calls — Dara Inc for years 2019 and 2018.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 50 expectedCount = 2	Failed	IN395 839 OUT4 361
×	TrackChristianityInTwoRegionsBFCL/parallel_34: two religion_history_track calls — Christianity in Egypt and Turkey, 100–1500 A.D.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 11 expectedCount = 2	Failed	IN25 919 OUT2 233
Tool Calling - Parallel Calls: Markets & Environment 3 passed10 failed	23.1%	IN675 554 OUT18 652
×	CalculateCagrForTwoInvestmentsBFCL/parallel_176: two calculate_cagr calls — (5000,7000,5) and (8000,12000,3).1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 2	Failed	IN48 643 OUT1 664
×	CalculateEmissionSavingsForSolarAndWindBFCL/parallel_186: two calculate_emission_savings calls — solar(12 months) and wind(8 months), Midwest region. Prompt states the Midwest region explicitly -> region required (accept "Midwest"/"Midwest region").1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN23 535 OUT779
×	CompareFutureValueWithAndWithoutContributionsBFCL/parallel_182: two finance_calculate_future_value calls — (5000,0.07,10, no contribution) and (3000,0.06,10, contribution 200). First scenario states "no additional contributions" -> null or 0.1 AI query 69 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 69 expectedCount = 2	Failed	IN414 595 OUT7 590
✓	ComputeEuclideanNormForThreePointsBFCL/parallel_183: three math_hypot calls — (5,7), (10,15), (20,25). z omitted or 0 (2D points).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN13 150 OUT344
×	EstimatePopulationsForTwoSpeciesAcrossTwoYearsBFCL/parallel_185: four estimate_population calls — Bengal Tigers/India and African Elephants/Kenya for 2021 and for 2023. NOTE: upstream ground truth's third entry lists year "" (omitted -> default 2020), which is an upstream error: the prompt explicitly asks for 2023 for "both these species". We tighten both 2023 estimates to year == 2023 (fair deterministic answer).1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4	Failed	IN20 135 OUT1 178
×	FindRootsForTwoQuadraticEquationsBFCL/parallel_184: two algebra_quadratic_roots calls — (3,7,2) and (5,-4,1).1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2	Failed	IN29 422 OUT1 229
×	GetClosingPricesForTwoCompaniesOnTwoDatesBFCL/parallel_178: four get_stock_price (by-date variant) calls — the 2x2 cross product of {Microsoft, Apple} x {2022-01-01, 2022-02-01} on NASDAQ. Upstream ground truth's first entry loosely lists company_name as [Microsoft, Apple]; the deterministic human answer is the four distinct (company, date) combinations, which we require. exchange NASDAQ (default) may be omitted.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN14 231 OUT885
×	GetDetailedAirQualityForThreeDaysBFCL/parallel_187: three get_air_quality calls — New York City, detail true, for 2023-05-05, 05-04, 05-03. Prompt asks for PM2.5/PM10/ozone/sources -> detail == true; "today/yesterday/day before" with today = May 5, 2023.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3	Failed	IN25 262 OUT1 000
✓	GetFourPriceTypesForTwoCompaniesBFCL/parallel_180: eight stock_price calls — {Microsoft, Apple} x {Open, Close, High, Low}, days 30. Prompt explicitly lists all four data types, so data_type must be set explicitly (required) to distinguish the eight calls; omission (default 'Close') cannot represent the four requested types.1 AI query 8 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL	Passed	IN9 442 OUT625
×	GetPricePerOunceForFourMetalsBFCL/parallel_177: four get_metal_price calls — gold, silver, platinum, palladium, measure 'ounce'. Prompt asks "price per ounce" -> measure == "ounce" required.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4	Failed	IN19 733 OUT776
×	GetStockPricesForFourCompaniesAndDurationsBFCL/parallel_181: four get_stock_prices calls — Apple(1 week), Microsoft(2 weeks), Amazon(3 weeks), Tesla(1 month). Each 'companies' argument is a single-element list.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN28 930 OUT582
✓	GetTrafficInfoForThreeLegsWithModesBFCL/parallel_188: three get_traffic_info calls — NY->LA (driving), LA->SF (bicycling), SF->NY (transit). First leg explicitly says "driving" -> mode == "driving" required (tightened; also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 763 OUT631
×	ProjectFutureValueForThreeStocksBFCL/parallel_175: three portfolio_future_value calls — AAPL(5000,0.07,5), MSFT(8000,0.06,7), AMZN(10000,0.08,10).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN19 713 OUT1 369
Tool Calling - Parallel Calls: Mathematics & Physics (set 1) 5 passed8 failed	38.5%	IN713 810 OUT13 441
×	AreaOfTwoRightTrianglesInMetersBFCL/parallel_68: two calculate_area calls — right triangles base/height 12x15 and 18x24 in meters. unit required (tool default is 'cm'; the prompt states meters/square meters).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 2	Failed	IN12 222 OUT480
×	CheckPlayerStatusAcrossThreeSeasonsBFCL/parallel_49: three player_status_check calls — player 3142 in RocketLeague for seasons 2017/2018/2019. Seasons are explicitly given, so each is required (default 2023 not applicable).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN17 875 OUT829
✓	CircumferenceOfFourCirclesInCmBFCL/parallel_65: four calculate_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt gives every radius in cm (e.g. "5cm") and the tool default unit is 'm', so a correct call must pass unit == 'cm'/'centimeter' (ground truth requires it on the first call and merely allows omission on the rest; we require it consistently since it is stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 217 OUT591
×	ConvertEuroAndPoundsToDollarsBFCL/parallel_59: two get_conversion_cost calls — 15000 Euro->dollars and 200 pounds->dollars.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN16 897 OUT684
×	FinalVelocityForFourDropsBFCL/parallel_81: four calculate_final_velocity calls — heights 10/20/15/25, initial_velocity 0 (stated), gravity 9.81 (default, may be omitted).1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 4	Failed	IN13 632 OUT382
✓	FindItalianAndJapaneseRestaurantsBFCL/parallel_52: two restaurant_finder calls — Italian in New York (preferences ['Vegetarian']) and Japanese in Los Angeles (delivery; ['Delivery'] or omitted since default is ['Delivery']).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN7 888 OUT239
×	FindRootsOfTwoQuadraticsAlgebraBFCL/parallel_62: two algebra_quadratic_roots calls — 3x^2+4x+2 and 5x^2-7x+3.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 461 OUT922
×	FindSoupAndVeganSaladRecipesBFCL/parallel_51: two recipe_search_find calls — Chicken Noodle Soup (diet omitted/default 'Keto') and a Vegan Salad (diet == 'Vegan', dish name accepts several phrasings).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN16 649 OUT442
×	GcdOfJohnAndMaryNumbersBFCL/parallel_80: two math_gcd calls — gcd(36,48) and gcd(60,96). Same tool as parallel_76.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 2	Failed	IN26 793 OUT624
✓	GcdOfTwoPairsBFCL/parallel_76: two math_gcd calls — gcd(45,60) and gcd(81,27).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 882 OUT146
×	GetPokemonMovesAndCheckSolarBeamBFCL/parallel_48: two PokemonGO_get_moves calls — Pikachu's move list (move omitted/default 'Run') and a check whether Bulbasaur can learn 'Solar Beam'.1 AI query 100 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.Completed Values: response.Completed = false	Failed	IN557 280 OUT6 878
✓	PrimeFactorsFormattedAndArrayBFCL/parallel_74: four get_prime_factors calls — numbers 4567 and 7890, each formatted true and false (user wants "a formatted string as well as an array").1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 164 OUT1 086
✓	SaveGameProgressEasyAndHardBFCL/parallel_50: two game_save_progress calls — stage 7 easy and stage 3 hard; level omitted/default 'user'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 850 OUT138
Tool Calling - Parallel Calls: Mathematics & Physics (set 2) 3 passed10 failed	23.1%	IN369 596 OUT12 937
×	AreaOfThreeCirclesInMetersBFCL/parallel_66: three geometry_area_circle calls — radii 5/10/15 in meters (default is meters, so units may be omitted).1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 18 expectedCount = 3	Failed	IN36 168 OUT1 202
✓	BookTwoHotelRoomsBFCL/parallel_57: two hotel_booking_book calls — Sheraton (NY, 2 adults/1 child) and Marriott (LA, 1 adult/2 children).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 387 OUT364
×	CheckAvailabilityForTwoHotelsBFCL/parallel_56: two hotel_booking_check_availability calls — Queens Hotel (Berlin) and Royal Hotel (Paris), each with check-in/out dates (several date formats accepted).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN18 051 OUT719
×	CircumferenceOfFourCirclesInMetersBFCL/parallel_70: four geometry_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt asks for output in meters and gives radii in m, but the tool default unit is 'cm', so a correct call must pass units == 'm'/'meters' (ground truth requires it on the first call and only allows omission on the rest; we require it consistently since it is explicitly stated).1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 4	Failed	IN13 140 OUT478
✓	DerivativeOfPolynomialAndItsDerivativeBFCL/parallel_71: two calculate_derivative calls — 3x^3-2x^2+5x-7 at x=4, then its derivative 9x^2-4x+5 at x=2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN12 284 OUT408
✓	FinalVelocityForThreeDragRaceRoundsBFCL/parallel_83: three final_velocity calls — (v0=0,a=5,t=10), (v0=10,a=7,t=8), (v0=20,a=4,t=12).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 516 OUT530
×	FindGroceryStoresByCriteriaBFCL/parallel_55: two grocery_store_find_by_criteria calls — NYC ['24 hours'] and San Diego ['Home Delivery'].1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2	Failed	IN27 712 OUT1 028
×	GetTwoCookingRecipesBFCL/parallel_53: two get_cooking_recipe calls — Lasagne Bolognese for 4 and Caesar Salad for 2.1 AI query 42 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 42 expectedCount = 2	Failed	IN154 678 OUT3 961
×	GetTwoExchangeRatesBFCL/parallel_58: two get_exchange_rate calls — USD->AUD and USD->CAD.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN17 269 OUT388
×	HighestCommonFactorOfFourPairsBFCL/parallel_77: four math_hcf calls — (45,60), (90,120), (36,48), (72,96). FLAGGED (see final report): the prompt is phrased as chaining ("use that result to find the HCF with...") but the ground truth treats it as four INDEPENDENT parallel calls on the literal pairs; a human might interpret it as sequential composition. Kept BFCL-faithful (four independent literal-pair calls) since parallel semantics preclude feeding one call's result into another.1 AI query 7 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 4	Failed	IN28 824 OUT1 543
×	OrderSameItemsFromTwoWholeFoodsLocationsBFCL/parallel_54: two whole_foods_order calls — same two items (order-independent) at downtown and uptown, size large.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 2	Failed	IN14 410 OUT238
×	PrimeFactorizeAsDictionaryBFCL/parallel_79: two prime_factorize calls — 4567 and 7890, return_type 'dictionary' (explicitly requested).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN17 369 OUT1 217
×	SolveTwoQuadraticEquationsBFCL/parallel_63: two solve_quadratic_equation calls — (5,6,1) and (3,2,1). Prompt names this function explicitly.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 788 OUT861
Tool Calling - Parallel Calls: Mathematics & Physics (set 3) 2 passed10 failed	16.7%	IN290 564 OUT12 029
×	AreaOfTwoCirclesInMetersBFCL/parallel_67: two geometry_calculate_area_circle calls — radii 5 and 10 in meters. unit required (tool default is 'cm'; the prompt states meters).1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 2	Failed	IN32 811 OUT1 178
×	AreaOfTwoTrianglesBFCL/parallel_69: two calculate_triangle_area calls — base/height 10x5 and 8x6.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2	Failed	IN27 349 OUT747
×	DerivativeOfTwoFunctionsAtValuesBFCL/parallel_73: two calculus_derivative calls — 3x^2+2x-1 at 5 (variable x, default) and 4y^3-3y^2+2y-1 at 3 (variable y).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN17 417 OUT740
×	DisplacementForThreeCarsBFCL/parallel_84: three calculate_displacement calls — (15,7,accel 3.5), (20,10,accel 2.0), (25,8,no acceleration). The third car travels "without any acceleration" so acceleration == 0 (also the default; omission accepted).1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 3	Failed	IN21 691 OUT744
✓	EuclideanNormForThreePointsBFCL/parallel_61: three math_hypot calls — (3,4) and (6,8) with z omitted/0, and (9,12,15).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 258 OUT264
×	FactorialOfThreeNumbersBFCL/parallel_60: three math_factorial calls — 5, 7, 9.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN12 647 OUT465
×	FinalSpeedForTwoFreeFallObjectsBFCL/parallel_85: two calculate_final_speed calls — initial_speed 0 for 10 s and initial_speed 5 for 7 s; gravity -9.81 (default, may be omitted).1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 2	Failed	IN23 858 OUT787
✓	GcdViaNumberTheoryBFCL/parallel_78: two number_theory_gcd calls — (45,60) and (81,63). Prompt names this function explicitly.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 026 OUT159
×	IntegrateTwoFunctionsWithTwoMethodsBFCL/parallel_72: four integrate calls — x3 on [2,5] and 2x2+3x-1 on [-1,3], each with trapezoid (default, may be omitted) and simpson (explicit).1 AI query 16 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 4	Failed	IN25 925 OUT2 033
×	PrimeFactorsOfThreeNumbersBFCL/parallel_75: three number_analysis_prime_factors calls — 45, 100, 150.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN12 668 OUT393
×	SolveFourQuadraticsWithRootTypeBFCL/parallel_64: four solve_quadratic calls — (2,5,3) all roots, (1,-3,2) real, (4,-7,3) all roots, (1,2,1) real. 'all' is the default so it may be omitted; 'real' is explicitly requested and required.1 AI query 40 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 40 expectedCount = 4	Failed	IN77 689 OUT3 941
×	VelocityForTwoCyclingDaysBFCL/parallel_82: two calculate_velocity calls — 120 km in 5 h and 150 km in 6 h; unit km/h (default, may be omitted).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN22 225 OUT578
Tool Calling - Parallel Calls: Mixed Domains 1 passed9 failed	10.0%	IN279 811 OUT34 049
×	CalculateBmiForTwoPeopleBFCL/parallel_4: two calculate_bmi calls — 6 ft / 80 kg and 5.6 ft / 60 kg.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN11 276 OUT306
×	CalculateEmForceForTwoTimeIntervalsBFCL/parallel_1: two calculate_em_force calls — same field/area, d_time 4 then 10.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2	Failed	IN25 225 OUT1 170
×	CalculateFactorialForThreeNumbersBFCL/parallel_7: three math_factorial calls — 5, 10, 15.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN16 117 OUT495
×	CalculateResistanceForCopperAndAluminumBFCL/parallel_2: two calculate_resistance calls — copper then aluminum, same length/area. Tightened per methodology: the prompt explicitly names "resistivity of copper", so the first call must pass resistivity == "copper" (ground truth also accepts omission via the 'copper' default, but an explicitly-stated value must be supplied). Aluminum is non-default → required.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN11 324 OUT771
×	CalculateSalesTaxForThreeCitiesBFCL/parallel_6: three calculate_sales_tax calls — Chicago/IL $30.45, Sacramento/CA $52.33, Portland/OR $11.23.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN16 645 OUT1 010
×	FetchPopulationForCitiesStateAndCountryBFCL/parallel_8: four database_us_census_get_population calls — NYC (city), LA (city), Alaska (state), USA (country); year defaults to 2000 (may be omitted). Requires area-type inference.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4	Failed	IN17 395 OUT987
×	FindMovieShowingsForTwoMoviesBFCL/parallel_9: two find_movie_showing calls near San Diego — Tenet at 5 pm and No Time To Die at 7:30 pm.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN15 276 OUT432
×	GetSequenceAnd3DForThreeProteinsBFCL/parallel_3: three protein_info_get_sequence_and_3D calls — HbA1c, normal + rat hemoglobin. Tightened per methodology: the prompt explicitly asks for "their 3D models", so model_3d == true is required (ground truth also accepts omission via the default of true).1 AI query 299 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 299 expectedCount = 3	Failed	IN125 098 OUT27 749
×	ListShowRatingsOnNetflixAndHuluBFCL/parallel_5: two streaming_services calls — Netflix['Friends'] and Hulu['The Office','Stranger Things'] (Hulu show order unordered), both sorted by rating. Must group shows per service.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 2	Failed	IN34 189 OUT989
✓	PlayTwoArtistsForGivenDurationsBFCL/parallel_0: two spotify_play calls — Taylor Swift for 20 min and Maroon 5 for 15 min.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN7 266 OUT140
Tool Calling - Parallel Calls: Physical & Life Sciences (set 1) 3 passed9 failed	25.0%	IN214 425 OUT13 100
×	ElectricFieldForTwoChargesBFCL/parallel_94: two calculate_electric_field calls — charge 5/distance 2 and charge 3/distance 4. "Conducted in a vacuum" = the default permitivity → may be omitted or 0.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 971 OUT423
×	ElectromagneticForceInTwoMediaBFCL/parallel_96: two electromagnetic_force calls — same charges 5,10 and distance 2, differing by medium_permittivity. Vacuum call (8.854e-12) equals the default → may be omitted or 8.854e-12; the second medium (5e-12) is non-default → required.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN18 361 OUT1 659
×	EstimateTravelTimeForThreeRoutesBFCL/parallel_93: three route_estimate_time calls — (1) NYC->Miami via Philadelphia/Washington D.C./Atlanta, (2) NYC->Miami skipping Atlanta (via Philadelphia/Washington D.C.), (3) Philadelphia-> Miami via Washington D.C. Stop orderings from ground truth accepted. FLAG parallel_93: BORDERLINE. Ground truth allows many stop-list variants, some including the start city itself as a stop; a competent human's stop set is not fully deterministic. Kept BFCL-faithful by accepting the natural stop orderings (start city excluded from stops).1 AI query 6 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN13 478 OUT864
×	FinalSpeedForTwoDroppedObjectsBFCL/parallel_88: two calculate_final_speed calls — v0 0/height 10 and v0 5/height 20. gravity stated as 9.8 (the default) so it may be omitted or 9.8 per ground truth.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 2	Failed	IN13 506 OUT465
×	FinalVelocityForTwoCarsBFCL/parallel_87: two calculate_final_velocity calls — Car A (v0 0, a 6, t 10), Car B (v0 20, a 4, t 15).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 037 OUT477
×	FinalVelocityFromDistanceForTwoObjectsBFCL/parallel_86: two kinematics_final_velocity_from_distance calls — accel 5/dist 100 and accel 10/dist 200. "Both objects start from rest" → initial_velocity may be omitted or 0 (default).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 980 OUT572
×	FindVeganRestaurantsInThreeCitiesBFCL/parallel_91: three vegan_restaurant_find_nearby calls — LA, SF, Seattle, each with operating_hours 22 (prompt: "open until at least 22:00", a non-default value → required).1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3	Failed	IN24 321 OUT925
✓	GenerateFourTravelItinerariesBFCL/parallel_90: four travel_itinerary_generator calls — Tokyo/7/200/urban, Paris/10/150/history, Sydney/5/100/nature, Rome/12/180/culture. Tokyo's "urban" equals the default → omit or "urban".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 791 OUT2 692
✓	GetDirectionsForRoundTripBFCL/parallel_89: three get_directions calls — SF->Palo Alto (fastest), Palo Alto->Golden Gate Bridge (scenic), Golden Gate Bridge->SF (fastest). route_type explicitly stated each time.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 516 OUT1 085
✓	MagneticFieldForTwoLoopsBFCL/parallel_95: two calculate_magnetic_field calls — current 10/radius 0.5 and current 15/radius 1.0. "Same as in free space" = default permeability → may be omitted or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 518 OUT544
×	ResonantFrequencyForTwoCircuitsBFCL/parallel_97: two calculate_resonant_frequency calls — L 0.005/C 1e-07/round 3 and L 0.007/C 2e-07/round 4. Rounding is stated explicitly → required.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 2	Failed	IN35 247 OUT1 616
×	ShortestDrivingDistanceRoundTripBFCL/parallel_92: three get_shortest_driving_distance calls — NYC->LA, LA->Miami, Miami->NYC, all in miles (prompt explicitly "in miles" → non-default unit required).1 AI query 21 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 21 expectedCount = 3	Failed	IN44 699 OUT1 778
Tool Calling - Parallel Calls: Physical & Life Sciences (set 2) 12 failed1 broken	0.0%	IN317 452 OUT29 874
×	AbsolutePressureAtTwoAltitudesBFCL/parallel_102: two calc_absolute_pressure calls — gauge 2.5 with atm 1 (default; may be omitted) and gauge 2.5 with atm 0.85 (non-default → required).1 AI query 20 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 20 expectedCount = 2	Failed	IN61 101 OUT1 881
×	BoilingMeltingPointsAcrossSeaLevelsBFCL/parallel_100: four get_boiling_melting_points calls — water@0, iron@1000, water@1000, iron@0. FLAG parallel_100: BORDERLINE. The prompt ("water and iron at sea levels of 0 and 1000 meters respectively") reads most naturally as TWO calls (water@0, iron@1000), but the BFCL ground truth is the full 2x2 cross product (FOUR calls). Kept BFCL-faithful to the ground truth; a competent human could reasonably answer with two calls, so this item's expected call count is not fully deterministic.1 AI query 20 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 20 expectedCount = 2	Failed	IN37 286 OUT1 510
×	CellInfoDetailedAndBriefBFCL/parallel_108: two biology_get_cell_info calls — neuron with detailed=true (prompt: "detailed description"), muscle with detailed=false ("less detailed", the default).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN22 980 OUT1 214
×	DensityForTwoSubstancesBFCL/parallel_101: two calculate_density calls (mass/volume schema) — 10 kg / 2 m³ and 15 kg / 3 m³. unit "kg/m³" equals the default → may be omitted or "kg/m³".1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN18 021 OUT498
×	ElectricFieldStrengthAtSeveralDistancesBFCL/parallel_98: four calculate_electric_field_strength calls — charge 2, distances 0.5/1/2 in vacuum (default medium), plus distance 1 in air. The distance-1 vacuum and distance-1 air calls are distinguished by medium. Vacuum equals the default → may be omitted or "vacuum"; "air" is required.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4	Failed	IN19 961 OUT1 705
×	EntropyChangeForTwoConditionsBFCL/parallel_103: two entropy_change_calculate calls — substance A, mass 2, temps 25->75 and 10->50. pressure 1 equals the default → may be omitted or 1.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN24 000 OUT1 437
×	FetchDnaSequenceThreeWaysBFCL/parallel_106: three fetch_DNA_sequence calls for id XYZ123 — default fasta/0, then genbank format, then 500 bp upstream. Defaults (fasta, upstream 0) may be omitted.1 AI query 18 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 18 expectedCount = 3	Failed	IN37 473 OUT1 212
×	FunctionLookupForTwoMoleculesBFCL/parallel_110: two cell_biology_function_lookup calls — ATP in mitochondria and DNA in nucleus, both with specific_function=true (prompt asks for a specific function within the organelle).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN23 320 OUT976
×	HeatCapacityForThreeConditionsBFCL/parallel_105: three calc_heat_capacity calls — 300K/2.5m³, 350K/2.5m³, 300K/1.5m³, all for air. "air" equals the default gas → may be omitted or "air".1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN13 700 OUT1 292
×	MolecularWeightForTwoCompoundsBFCL/parallel_111: two calculate_molecular_weight calls — Glucose (C6H12O6) and Sucrose (C12H22O11), both in grams/mole (prompt explicitly requests that unit; ground truth also accepts "g/mol").1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN23 270 OUT1 232
×	PhaseChangeEnergyForWaterBFCL/parallel_99: four thermo_calculate_energy calls — 500 g water, phase melting/freezing/ vaporization/condensation. Substance "water" equals the default → may be omitted or "water".1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN14 186 OUT1 401
!	ProteinSequencesForTwoGenesTwoSpeciesBFCL/parallel_107: four get_protein_sequence calls — BRCA1 and BRCA2 in Homo sapiens (default species, may be omitted) and in Pan troglodytes (non-default → required).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.Completed Values: response.Completed = false	Broken	IN8 712 OUT14 724
×	ProteinsInThreeCompartmentsBFCL/parallel_109: three cellbio_get_proteins calls — nucleus/mitochondria/cytoplasm, each with include_description=true (prompt asks for "a brief description of each protein").1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN13 442 OUT792
Tool Calling - Parallel Calls: Physical & Life Sciences (set 3) 12 failed1 broken	0.0%	IN261 759 OUT24 035
×	AnalyzeDnaForThreeMutationTypesBFCL/parallel_114: six analyze_dna_sequence calls — two sequences ("AGCTTAGCTA" and "AGCTTAGGCTA") against reference "AGCTTAGCTA", each checked for insertion (default, may be omitted), deletion, and substitution mutations.1 AI query 3 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 6	Failed	IN8 720 OUT738
×	DiabetesPredictionForFourPeopleBFCL/parallel_113: four diabetes_prediction calls for four people with distinct weight/height/activity.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4	Failed	IN20 275 OUT1 340
×	EntropyChangeIsothermalAndNotBFCL/parallel_104: two calculate_entropy_change calls — same 300->350 K and heat_capacity 4.18, once isothermal (the default true, may be omitted) and once not isothermal (false, required).1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2	Failed	IN29 620 OUT1 916
×	ForestGrowthForTwoRegionsBFCL/parallel_120: two forest_growth_forecast calls — Amazon Rainforest over 10 years and Boreal Forests of Canada over 20 years, excluding human impact (false = the default, may be omitted).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN18 147 OUT776
!	GenerateFourDnaSequencesBFCL/parallel_123: four generate_DNA_sequence calls — length 500 with a single-nucleotide preference of A, T, C, G respectively.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.Completed Values: response.Completed = false	Broken	IN19 827 OUT10 495
×	GeneticSimilarityInTwoFormatsBFCL/parallel_115: four genetics_calculate_similarity calls — human vs chimpanzee and human vs gorilla, each in percentage (default, may be omitted) and fraction. species1 accepts "human"/"Human".1 AI query 20 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 20 expectedCount = 4	Failed	IN33 650 OUT1 759
×	GenotypeFrequenciesForThreeGenotypesBFCL/parallel_116: three calculate_genotype_frequency calls — allele frequency 0.7, genotypes AA/Aa/aa.1 AI query 15 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 15 expectedCount = 3	Failed	IN31 085 OUT1 180
×	IdentifyFourBirdsBFCL/parallel_119: four identify_bird calls — small blue/forest (default size, may be omitted), large black/lake, medium brown/desert, large green/tropical rainforest.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4	Failed	IN20 135 OUT1 919
×	MutationTypeForTwoSnpsBFCL/parallel_112: two mutation_type_find calls — rs123456 in Homo sapiens (default species, may be omitted) and rs7891011 in Canis lupus familiaris (non-default → required).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 202 OUT438
×	PopulationDensityForChinaTwoYearsBFCL/parallel_117: two calculate_density calls (country/population schema) — China in 2000 (pop 1.267 billion) and 2010 (pop 1.341 billion), land area 9.597 million km² both times.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN21 182 OUT719
×	PrecipitationStatsForThreeTimeFramesBFCL/parallel_118: three ecology_data_precipitation_stats calls — Amazon rainforest for six_months, year, five_years.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3	Failed	IN13 520 OUT1 021
×	TurtlePopulationForTwoYearsBFCL/parallel_121: two ecology_get_turtle_population calls — Galapagos Islands in 2015 and 2020, both with species info requested (species=true; years stated explicitly → required).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN13 199 OUT753
×	VehicleEmissionsForThreeVehiclesBFCL/parallel_122: three calculate_vehicle_emission calls — 15000 miles each: gas with default factor 1.4 (may be omitted), diesel with factor 2.7, EV with factor 0.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN19 197 OUT981
Tool Calling - Parallel Calls: Services & Lookups 1 passed11 failed	8.3%	IN205 096 OUT8 469
×	CalculateShortestAndScenicDistancesForTwoLegsBFCL/parallel_190: four calculate_shortest_distance calls — {NYC->LA, LA->Miami} x {Shortest, Scenic}.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN13 424 OUT763
×	FetchNewsForTwoTopicsInEuropeBFCL/parallel_192: two get_news calls — Climate Change and Artificial Intelligence, 5 articles each, region Europe. Region "Europe" explicitly requested -> required (accept "Europe"/"EU").1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 458 OUT761
×	FindLibrariesWithFacilitiesInThreeCitiesBFCL/parallel_191: three public_library_find_nearby calls — NY[Reading Room,Fiction], LA[Wi-Fi,Children Section], Chicago[Cafe,Reading Room]. Facilities compared as an unordered pair.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN18 179 OUT826
×	FindParksWithAmenitiesInThreeCitiesBFCL/parallel_189: three parks_find_nearby calls — NY[Tennis Court,Picnic Area], LA[Playground,Running Track], Chicago[Tennis Court,Playground]. Amenities compared as an unordered pair.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN18 201 OUT990
×	FindUpcomingEventsInThreeCitiesBFCL/parallel_194: three event_finder_find_upcoming calls — jazz/LA(14d), rock/Chicago(10d), classical/Boston(7d). All day windows explicitly stated -> days_ahead required (7 for Boston, which also equals the default).1 AI query 21 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 21 expectedCount = 3	Failed	IN43 983 OUT1 779
×	GetCurrentWeatherForFourCitiesBFCL/parallel_199: four get_current_weather calls — New York, Los Angeles, London, Tokyo, with both temperature and humidity included. Prompt explicitly asks to include both -> flags == true (also the default).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4	Failed	IN13 349 OUT621
×	GetDetailedBriefsForTwoMoviesBFCL/parallel_195: two movie_details_brief calls — Inception and The Dark Knight, extra_info true. Prompt asks for Director/Cast/Awards -> extra_info == true required.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN22 020 OUT711
×	GetLawsuitDetailsForTwoCasesBFCL/parallel_196: two get_lawsuit_details calls — case 12345 (NY Supreme Court, with verdict) and case 67890 (LA Superior Court, no verdict). "would like the verdict" -> true; "do not need" -> null/false.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 680 OUT462
×	GetLawsuitInfoForTwoCasesByYearBFCL/parallel_197: two lawsuit_info calls — 12345ABC (2018, New York) and 67890XYZ (2019, California). Years are explicitly stated -> required. First location "New York" may be omitted (matches the default).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN17 443 OUT579
×	GetStockPricesForTwoCompaniesOverDaysBFCL/parallel_179: two get_stock_price (by-days variant) calls — Apple on NASDAQ for 10 days, Microsoft on NYSE for 15 days. Both exchanges are explicitly named -> required (NASDAQ / NYSE). The tool's 'company' field is a ticker symbol; company names are accepted as synonyms.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 464 OUT329
×	SearchLawsuitsForTwoEntitiesInCaliforniaBFCL/parallel_198: two lawsuit_search calls — Google/Santa Clara and Facebook/San Mateo, state California. Prompt states "both in the state of California" -> state required (accept "California"/"CA").1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2	Failed	IN12 518 OUT344
✓	SendTwoEmailsWithCcAndBccBFCL/parallel_193: two send_email calls — Project Update to john.doe (cc manager, bcc hr) and Meeting Reminder to jane.doe (cc assistant, bcc hr). Bodies matched exactly.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL	Passed	IN8 377 OUT304
Tool Calling - Parallel Calls: Statistics & Finance 1 passed12 failed	7.7%	IN368 633 OUT19 234
×	CalculateBinomialProbabilitiesForBasketballShotsBFCL/parallel_163: three calc_binomial_prob calls — (10,6,0.6), (10,6,0.5), (15,6,0.5).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN20 047 OUT1 500
×	CalculateCompoundInterestForThreePeriodsBFCL/parallel_170: three calculate_compound_interest calls — principal 5000, rate 0.025, time 2/3/5. Prompt states "compounded quarterly, 4 times in a year" -> n == 4 required (tightened; also ground truth).1 AI query 27 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 27 expectedCount = 3	Failed	IN64 541 OUT2 285
×	CalculateDiscountedCashFlowForTwoBondsBFCL/parallel_169: two calculate_discounted_cash_flow calls — (50,5,0.05) and (60,7,0.04). Prompt explicitly states "face value of the bond is $1000" for both -> face_value == 1000 required (ground truth also accepts omission via the default, but an explicitly-stated value must be supplied).1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 2	Failed	IN50 722 OUT2 569
×	CalculateEventProbabilitiesForCardDeckBFCL/parallel_162: three probability_of_event calls — Ace 4/52 as decimal, heart 13/52 as decimal, red card 26/52 as ratio. format_as_ratio: "as a decimal" -> false or omitted; "as a ratio" -> true (required).1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 18 expectedCount = 3	Failed	IN39 867 OUT1 437
×	CalculateQuarterlyDividendForThreeScenariosBFCL/parallel_168: three finance_calculate_quarterly_dividend_per_share calls — (5M,2M), (6M,2M), (6M,2.5M).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3	Failed	IN20 201 OUT952
×	CalculateReturnOnEquityForTwoCompaniesBFCL/parallel_171: two calculate_return_on_equity calls — (1M,5M,200k) and (2M,10M, no dividends). Second company "did not pay any dividends" -> dividends_paid null or 0.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN24 700 OUT1 020
×	CalculateRoiForThreeStocksBFCL/parallel_174: three calculate_return_on_investment calls — (150,180,20), (200,210,30), (250,300,40). Dividends are explicitly stated per stock -> required.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3	Failed	IN26 116 OUT1 188
✓	ComputeRSquaredForTwoRegressionModelsBFCL/parallel_167: two linear_regression_get_r_squared calls on the same CSV — swap education/credit_score as dependent variable. Independent-variable order follows the prompt/ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN8 934 OUT557
×	PredictFutureValueForTwoInvestmentsBFCL/parallel_172: two finance_predict_future_value calls — (5000,0.05,10) and (7000,0.04,15). Prompt states "compound interest annually" -> compounding_periods_per_year == 1 required (tightened; also the default).1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2	Failed	IN30 664 OUT1 370
×	PredictHousePricesForThreeHousesBFCL/parallel_166: three predict_house_price calls — NY 2500/3/2000, LA 3000/3/2005, Chicago 2000/2/1995. The second house is described as "similar" with only area/year/location changed, so rooms == 3 per ground truth.1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 3	Failed	IN25 386 OUT1 144
×	PredictProfitForTwoFundsBFCL/parallel_173: two investment_predictProfit calls — (5000,0.07,5) and (8000,0.05,7).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2	Failed	IN18 953 OUT1 165
×	RunChiSquaredTestForTwoContingencyTablesBFCL/parallel_164: two chi_squared_test calls — tables [45,55,35,65] and [30,70,50,50], alpha 0.05. Prompt states "significance level of 0.05 for both" -> alpha == 0.05 required (tightened). Table order is fixed.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2	Failed	IN24 405 OUT2 002
×	RunTTestAtTwoSignificanceLevelsBFCL/parallel_165: two t_test calls — same datasets, alpha 0.05 then 0.01 (both stated -> required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 2	Failed	IN14 097 OUT2 045
Tool Calling - Simple: Arts, Venues & Music (set 1) 8 failed	0.0%	IN1 193 334 OUT15 312
×	SimpleConcertDetailsBeyonceBFCL/simple_python_286 (concert.get_details): artist=Beyonce, location=San Diego, date=April 2022.1 AI query 17 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 17	Failed	IN80 018 OUT1 621
×	SimpleConcertInfoPopNyBFCL/simple_python_284 (concert_info.get): location=New York, date=next month, genre=Pop.1 AI query 47 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 47	Failed	IN324 230 OUT3 981
×	SimpleGenerateMelodyC4BFCL/simple_python_291 (music_generator.generate_melody): key=C, start_note=C4, length=16, tempo=120 or omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3	Failed	IN14 040 OUT267
×	SimpleKeySignatureCsharpBFCL/simple_python_298 (music_theory.key_signature): key='C#', scale_type=major or omitted.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.Completed Values: response.Completed = false	Failed	IN347 516 OUT3 917
×	SimpleMuseumHoursLouvreBFCL/simple_python_273 (museum.get_hours): name='Louvre Museum', location=Paris, day omitted or Monday.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5	Failed	IN20 543 OUT339
×	SimplePaintNeededWall30x12BFCL/simple_python_259 (calculate_paint_needed): coverage_rate=400, length=30, height=12.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4	Failed	IN16 981 OUT368
×	SimpleReligionHistoryChristianity16cBFCL/simple_python_252 (get_religion_history): religion=Christianity, century=16, sort_by=importance, count=10.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.Completed Values: response.Completed = false	Failed	IN373 210 OUT4 330
×	SimpleSculptureInfoPlensaBFCL/simple_python_263 (get_sculpture_info): artist_name='James Plensa', detail=true.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4	Failed	IN16 796 OUT489
Tool Calling - Simple: Arts, Venues & Music (set 2) 8 failed	0.0%	IN465 485 OUT8 857
×	SimpleComposeMelodyCFGBFCL/simple_python_292 (compose_melody): progression=[C,F,G], measures=4, instrument=Piano or omitted.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10	Failed	IN42 074 OUT883
×	SimpleConcertFindDetailsWeekndBFCL/simple_python_290 (concert.find_details): artist=The Weeknd, month=December, year=2022 or omitted.1 AI query 35 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 35	Failed	IN206 966 OUT2 820
×	SimpleFindConcertRockChicagoBFCL/simple_python_285 (find_concert): location=Chicago, price=100, genre=Rock.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10	Failed	IN41 551 OUT678
×	SimpleMuseumInfoMetBFCL/simple_python_274 (museum_info): museum_name=Metropolitan Museum of Art, info_type=opening_hours or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5	Failed	IN20 501 OUT401
×	SimpleMusicalScaleCsharpBFCL/simple_python_299 (musical_scale): key='C#'/'C sharp', scale_type=major or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5	Failed	IN21 597 OUT373
×	SimplePaintRequirementWallExclusionBFCL/simple_python_260 (paint_requirement.calculate): area {width=20,height=12}, paint_coverage=350, exclusion {type=window, area=15}.1 AI query 18 tool callscategory:ToolCallinglevel:Hardsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 18	Failed	IN91 397 OUT2 271
×	SimpleReligionHistoryChristianity300To400BFCL/simple_python_254 (get_religion_history): religion=Christianity, start_year=300, end_year=400, event_type=all or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5	Failed	IN20 810 OUT785
×	SimpleSculptureDetailsDavidBFCL/simple_python_264 (sculpture.get_details): artist=Michelangelo, title=David, detail=size.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5	Failed	IN20 589 OUT646
Tool Calling - Simple: Arts, Venues & Music (set 3) 1 passed8 failed	11.1%	IN885 787 OUT12 316
✓	SimpleChordProgressionKeyBFCL/simple_python_297 (music.theory.chordProgression): progression=[I,V,vi,IV], bools any.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL	Passed	IN6 697 OUT354
×	SimpleCompoundInterest10kBFCL/simple_python_269 (calculate_compound_interest): principle=10000, interest_rate=0.05, time=10, compounds_per_year=1 or omitted.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8	Failed	IN35 468 OUT1 021
×	SimpleConcertSearchClassicalLaBFCL/simple_python_287 (concert.search): genre=classical, location=Los Angeles/LA, date='this weekend', price_range=cheap.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.Completed Values: response.Completed = false	Failed	IN377 045 OUT4 579
×	SimpleCreateMixCMajorBFCL/simple_python_293 (music_composer.create_mix): scale='C Major', note_duration=quarter, track_length=180.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2	Failed	IN10 262 OUT229
×	SimpleGenerateCircleImageRedBFCL/simple_python_256 (generate_circle_image): radius=50, color=Red, background=white or omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2	Failed	IN10 196 OUT127
×	SimpleMuseumWorkingHoursLouvreBFCL/simple_python_276 (museum_working_hours.get): museum=Louvre, location=Paris, day=Monday or omitted.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11	Failed	IN50 877 OUT914
×	SimpleRetrieveReligionInfoBuddhismBFCL/simple_python_253 (retrieve_religion_info): religion_name=Buddhism, detail_level=full.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6	Failed	IN25 764 OUT1 341
×	SimpleSculptureSearchChicago19cBFCL/simple_python_265 (sculpture_search): location=Chicago, time_frame='19th century', material=all or omitted.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL Assertion expression evaluated to false: response.Completed Values: response.Completed = false	Failed	IN359 297 OUT3 501
×	SimpleTimeDifferenceSfSydneyBFCL/simple_python_250 (get_time_difference): place1=San Francisco/SF, place2=Sydney.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2	Failed	IN10 181 OUT250

Tool Calling - Irrelevance: Architecture & Artwork 7 passed2 failed

77.8%

IN416 262 OUT6 892

✓

NoToolForApartmentRentCostBFCL/irrelevance_158: apartment rental cost asked; only a new-construction cost estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 463 OUT43

×

NoToolForBestJazzSingerBFCL/irrelevance_164: best Jazz singer asked; only a top-instrument-players search (singers are not instrument players) is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2

Failed

IN10 575 OUT274

✓

NoToolForCathedralCeilingDesignBFCL/irrelevance_157: how to design a cathedral ceiling asked; only a lookup of facts about an existing building is available (retrieves data, does not advise on design) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 459 OUT513

✓

NoToolForHolyBookDefinitionBFCL/irrelevance_144: definition of a holy book asked; only a search-within-a-named-holy-book tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 457 OUT93

✓

NoToolForLectureTeacherBFCL/irrelevance_167: teacher of upcoming lectures asked; only an upcoming-concert details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 460 OUT34

✓

NoToolForMostFamousSculptureBFCL/irrelevance_153: most famous sculpture asked; only a sculpting-tools lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 465 OUT411

✓

NoToolForPianoMaintenanceTipsBFCL/irrelevance_166: piano maintenance tips asked; only an instrument rental-prices lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 460 OUT563

×

NoToolForScreamArtistBFCL/irrelevance_159: artist behind 'The Scream' asked. NOTE: the only tool (artwork_search) requires a museum_location that the user did not provide, so it cannot be correctly called; BFCL labels this irrelevant. Kept faithfully with ground truth = no tool call (flagged as borderline).1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN381 458 OUT4 838

✓

NoToolForSculptureRockTypeBFCL/irrelevance_152: which rock suits a garden sculpture asked; only a sculpture 3D-model creator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 465 OUT123

Tool Calling - Irrelevance: Buildings, Sculpture & Music 6 passed1 failed1 skipped

85.7%

IN26 437 OUT1 600

–

NoToolForCelloInstrumentTypeBFCL/irrelevance_165: instrument classification of a cello asked. NOTE: BFCL labels this irrelevant, but the only tool (get_instrument_info with detail=type) arguably answers it; flagged for review as a borderline BFCL item. Kept faithfully with ground truth = no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_instrument_info' with detail='type' answers 'what type of instrument is a cello' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForDeclarationSignersBFCL/irrelevance_137: who signed the declaration asked; only a historical-event DATE lookup is available (returns a date, not the signatories) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 284 OUT251

✓

NoToolForEiffelTowerHeightBFCL/irrelevance_156: Eiffel Tower height asked; only an architecture-plan generator (creates a new design, does not retrieve facts) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 288 OUT60

✓

NoToolForFamousDancersBFCL/irrelevance_162: famous dancers asked; only a museum-visual-artists lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 290 OUT244

✓

NoToolForReformationInitiatorBFCL/irrelevance_145: who initiated the Reformation asked; only a religious-event YEAR lookup is available (returns a year, not a person) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 283 OUT80

✓

NoToolForScientistDefinitionBFCL/irrelevance_143: definition of "scientist" asked; only a named-historical-figure info lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 280 OUT344

✓

NoToolForSellingGuitarBFCL/irrelevance_163: how to sell a guitar asked; only an instrument-tuning tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 285 OUT338

×

NoToolForSimilarBlueColorBFCL/irrelevance_148: a color similar to blue asked; only a complementary-color tool (opposite, not similar) is available -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN6 727 OUT283

Tool Calling - Irrelevance: Cell Biology & Genetics 8 passed2 skipped

100.0%

IN30 331 OUT2 848

✓

NoToolForClottingBloodCellsBFCL/irrelevance_32: blood cell type responsible for clotting asked; only a nucleus-count-based cell-type classifier is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 792 OUT62

✓

NoToolForCommonAncestorBFCL/irrelevance_41: common ancestor of two animals asked (phylogenetics knowledge); only a fibonacci-sequence calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 791 OUT245

✓

NoToolForEvolutionaryAlgorithmInGameAIBFCL/irrelevance_43: how to apply an Evolutionary Algorithm in game AI asked (advice); only a creature-evolution simulator is available (wrong context).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 792 OUT1 044

✓

NoToolForGeneVariantFrequencyBFCL/irrelevance_36: gene-variant frequency in a population asked; only a dominant-trait-from-alleles tool is available (the true-match variant-frequency tool is deliberately isolated in A4).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 797 OUT355

–

NoToolForIdentifyingGeneticSequenceBFCL/irrelevance_33: interpret a genetic code sequence asked; only a species-identification-from-sequence tool is available (returns a species, not what the sequence means).0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'identify_species' (identifies a species from a genetic sequence) plausibly answers the query about a DNA sequence, so a competent model may call it - BFCL-mislabeled/borderline-relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForIncreasingDeerPopulationBFCL/irrelevance_39: how-to advice on growing an animal population asked; only a population-growth CALCULATOR (needs numeric rates) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 791 OUT246

✓

NoToolForLionMatingProcessBFCL/irrelevance_35: mating process of a species asked; only a dominant/recessive genetic-traits lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 788 OUT286

✓

NoToolForMitochondriaCountBFCL/irrelevance_29: number/count of mitochondria asked; only a cell-part FUNCTION lookup is available (wrong query direction).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 792 OUT198

✓

NoToolForPandaEvolutionaryHistoryBFCL/irrelevance_42: evolutionary history asked (a narrative fact); only a biodiversity-index calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 788 OUT412

–

NoToolForWhaleGeneSequenceBFCL/irrelevance_44: the actual gene sequence of a species asked; only a gene-sequence GENERATOR (needs numeric mutation rate + duration) is available.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'gene_sequencer' (generate gene sequences for evolutionary changes) directly answers the query - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Irrelevance: Color, Geometry & Population 10 passed1 failed

90.9%

IN424 400 OUT5 399

✓

NoToolForColorInComputerVisionBFCL/irrelevance_131: color meaning in computer vision asked; only a color-in-personality-psychology analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 714 OUT116

✓

NoToolForDeclarationSigningDateBFCL/irrelevance_135: signing date asked; only date-arithmetic (add days) and age tools are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 712 OUT45

✓

NoToolForElectricityDiscovererBFCL/irrelevance_140: who discovered electricity asked; only an electric-current calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 708 OUT211

✓

NoToolForHexColorMixBFCL/irrelevance_146: mixing two hex colors asked; only a prophet-details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 724 OUT315

✓

NoToolForIndependenceSigningDateBFCL/irrelevance_138: signing date asked; only an age calculator (needs birthdate + current date) and a date-adder are available, neither retrieves an event date -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 712 OUT69

✓

NoToolForMostVisitedMarketBFCL/irrelevance_161: most visited market asked; only a museum visitation-statistics lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 714 OUT209

✓

NoToolForNbaFinalWinnerBFCL/irrelevance_133: NBA final winner asked; only a historical-battle details lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 715 OUT50

✓

NoToolForPicassoPaintingYearBFCL/irrelevance_151: year of a Picasso painting asked; only a sculpture-dimensions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 713 OUT130

✓

NoToolForRelativityAuthorBFCL/irrelevance_142: who proposed special relativity asked; only a chemical-element properties lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 717 OUT15

×

NoToolForSculptureCreatorBFCL/irrelevance_155: sculpture creator asked; only a max-seating-capacity calculator is available -> no tool fits.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN387 255 OUT4 095

✓

NoToolForWhichColorsToMixBFCL/irrelevance_150: which colors to mix for a shade asked; only an RGB-to-Pantone converter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 716 OUT144

Tool Calling - Irrelevance: Cooking, Currency & Markets 7 passed1 failed

87.5%

IN40 007 OUT1 599

×

NoToolForBakerySourdoughBFCL/irrelevance_223: a BAKERY selling sourdough asked; only a grocery-shop product finder available -> no tool fits.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 4

Failed

IN17 295 OUT432

✓

NoToolForChickenRecipeBFCL/irrelevance_211: the RECIPE to cook chicken asked; only a cook-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 243 OUT372

✓

NoToolForChocolateCakeIngredientsBFCL/irrelevance_220: cake INGREDIENTS asked; only a cooking-TEMPERATURE tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 246 OUT144

✓

NoToolForCurrencyToGbpBFCL/irrelevance_233: CURRENCY conversion asked; only a WEATHER tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 248 OUT23

✓

NoToolForHowToBoilAnEggBFCL/irrelevance_212: best WAY to boil an egg asked; only a boiling-TIME calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 245 OUT239

✓

NoToolForLasagnaRecipeBFCL/irrelevance_219: the RECIPE for lasagna asked; only a recipe-CALORIE calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 243 OUT22

✓

NoToolForLegExercisesBFCL/irrelevance_221: leg EXERCISES asked; only a grocery-list generator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 243 OUT241

✓

NoToolForPetStoreBFCL/irrelevance_224: a PET store asked; only a GROCERY-store locator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 244 OUT126

Tool Calling - Irrelevance: Dates & Elements 9 passed1 failed

90.0%

IN47 876 OUT1 672

✓

NoToolForHydrogenPropertiesBFCL/irrelevance_141: chemical properties of Hydrogen asked; only a scientist-contributions lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 628 OUT362

✓

NoToolForLargestPlanetBFCL/irrelevance_139: largest planet asked; only a star-info lookup (needs a star name) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 628 OUT135

✓

NoToolForMostImportantProphetBFCL/irrelevance_147: most important prophet asked; only a two-color mixer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 628 OUT121

×

NoToolForMuseumVisitFrequencyBFCL/irrelevance_160: museum member visit frequency asked; only a single most-frequent-visitor lookup is available -> no tool fits.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 3

Failed

IN15 219 OUT414

✓

NoToolForPantoneOfSkyBlueBFCL/irrelevance_149: Pantone code of sky blue asked; only a paint-proportions calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 630 OUT222

✓

NoToolForPersonalityTypeBFCL/irrelevance_130: user asks what personality type they are; only a big-five calculator (which needs the user's rated answers) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 626 OUT105

✓

NoToolForStadiumSeatingCapacityBFCL/irrelevance_154: stadium seating capacity asked; only a sculpture-creator lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 629 OUT86

✓

NoToolForVicePresidentBFCL/irrelevance_136: Vice President asked; only a US-president-by-year lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 628 OUT63

✓

NoToolForWaterlooCasualtiesBFCL/irrelevance_132: casualty count of a battle asked; only a historical-event date lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 630 OUT128

✓

NoToolForWorldCupWinnerBFCL/irrelevance_134: sports World Cup winner asked; only a historical-battle outcome predictor is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 630 OUT36

Tool Calling - Irrelevance: Electromagnetism, Thermo & Data 10 passed

100.0%

IN34 065 OUT1 865

✓

NoToolForElectronRestMassBFCL/irrelevance_23: mass of an electron asked (a knowledge fact); only magnetic-field / current tools are available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 399 OUT64

✓

NoToolForFlightDurationBFCL/irrelevance_20: flight duration asked; only a currency converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 402 OUT88

✓

NoToolForFreezingPointBFCL/irrelevance_25: freezing point asked; only boiling-point calculators are available (wrong phase transition).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 409 OUT285

✓

NoToolForGasAmountGeneratedBFCL/irrelevance_26: amount of gas generated asked; only a gas-PRESSURE calculator is available (wrong quantity).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 422 OUT700

✓

NoToolForGiniCoefficientBFCL/irrelevance_48: Gini coefficient asked; only a GDP-fetch tool is available (different economic indicator).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 402 OUT108

✓

NoToolForHeatingWaterBFCL/irrelevance_27: heat energy to warm water asked. NOTE: the original tool (calculate_heat) is a fair match for this query (only the well-known specific heat is missing), which would make "no call" ambiguous; per human-baseline fairness it is deliberately NOT shown here (kept as a distractor in A3), so no visible tool fits and "no call" is unambiguously correct.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 413 OUT272

✓

NoToolForHexagonSidesBFCL/irrelevance_28: number of sides of a hexagon asked (basic geometry knowledge); only a boiling-point tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 400 OUT8

✓

NoToolForSymbolicMagneticFieldBFCL/irrelevance_22: magnetic field asked with SYMBOLIC values (distance 'r', current 'I'); the two magnetic-field tools require concrete numeric distance + current, so there is nothing to pass -> no fit.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 413 OUT206

✓

NoToolForTimeDifferenceBFCL/irrelevance_49: time-zone difference asked; only a geo-coordinate lookup tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 402 OUT118

✓

NoToolForWorldSeriesWinnerBFCL/irrelevance_53: sports trivia asked; only a SQL database-query runner is available (no database/query provided).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 403 OUT16

Tool Calling - Irrelevance: Entertainment, Sports & Knowledge 9 passed

100.0%

IN31 428 OUT738

✓

NoToolForBasketballResultBFCL/irrelevance_84: basketball game result asked; only a stock-price retrieval tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 492 OUT39

✓

NoToolForBestSoccerPlayerBFCL/irrelevance_79: best current soccer player asked; only a stock-market prediction tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 490 OUT37

✓

NoToolForCaliforniaGovernorBFCL/irrelevance_87: current Governor of California asked; only a court-case filing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 488 OUT56

✓

NoToolForCrimeThrillerMovieListBFCL/irrelevance_88: best crime-thriller movies of 2020 asked; only a document forgery detector is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 497 OUT193

✓

NoToolForFifaWorldCupWinnerBFCL/irrelevance_80: FIFA World Cup 2010 winner asked; only a stock-ticker lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 493 OUT49

✓

NoToolForFrenchTranslationBFCL/irrelevance_61: French translation of a word asked; only an SVM-model tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 492 OUT14

✓

NoToolForHorrorMovieListBFCL/irrelevance_77: horror movie recommendations asked; only an investment future-value tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 491 OUT296

✓

NoToolForPresidentialElectionWinnerBFCL/irrelevance_85: 2020 presidential election winner asked; only a criminal-case detail lookup is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 493 OUT27

✓

NoToolForSportsNewsBFCL/irrelevance_76: latest sports news asked; only a portfolio investment-trend analyzer is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 492 OUT27

Tool Calling - Irrelevance: Finance & Economics 7 passed1 failed

87.5%

IN33 742 OUT1 112

✓

NoToolForAverageHousePriceBFCL/irrelevance_69: market-wide average house price asked; only a regression predictor (needs a trained model + feature vector, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 348 OUT89

✓

NoToolForCompoundInterestBFCL/irrelevance_70: compound-interest computation asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 374 OUT264

✓

NoToolForGoldPriceBFCL/irrelevance_78: today's gold spot price asked; only a bond-price calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 350 OUT63

✓

NoToolForHorrorMovieCatalogBFCL/irrelevance_81: horror movie recommendations asked; only a historical stock-price fetcher is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 351 OUT297

×

NoToolForLossProjectionBFCL/irrelevance_73: a loss projection asked; only a revenue-prediction tool is available (revenue != loss).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2

Failed

IN10 253 OUT195

✓

NoToolForPoundToKilogramBFCL/irrelevance_75: a mass unit conversion asked; only a portfolio-returns calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 349 OUT19

✓

NoToolForProfitMarginBFCL/irrelevance_71: profit margin asked; only an ROI calculator is available (ROI != margin) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 371 OUT144

✓

NoToolForStockNewsBFCL/irrelevance_82: stock news asked; only a capital-gains calculator is available (gains != news) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 346 OUT41

Tool Calling - Irrelevance: Finance Near-Matches 9 passed

100.0%

IN30 939 OUT2 075

✓

NoToolForBitcoinPriceBFCL/irrelevance_56: current bitcoin price asked; only a generic database record-select tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 428 OUT45

✓

NoToolForBookstoreLookupBFCL/irrelevance_55: which online bookstore sells a title asked; only an add-to-cart tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 433 OUT55

✓

NoToolForBusinessRateOfReturnBFCL/irrelevance_74: rate of return asked; only a net-profit calculator is available (profit != rate of return).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 449 OUT584

✓

NoToolForCoinFlipProbabilityBFCL/irrelevance_66: probability of a coin landing heads asked; only a standard-deviation tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 433 OUT42

✓

NoToolForCurrentInterestRateBFCL/irrelevance_83: the current market interest rate asked; only a mortgage-payment calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 427 OUT77

✓

NoToolForExternalRateOfReturnBFCL/irrelevance_72: EXTERNAL rate of return asked; only an internal-rate-of-return (IRR) tool is available (ERR != IRR, even though cash flows are given) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 457 OUT353

✓

NoToolForFinalPriceWithoutBasePriceBFCL/irrelevance_60: final price after discount+tax asked, but the original price is never given, so the (domain-matching) calculateFinalPrice cannot be correctly called -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 447 OUT181

✓

NoToolForFrontendDevelopmentBFCL/irrelevance_62: how to build an e-commerce frontend asked; only a recommender-model builder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 433 OUT684

✓

NoToolForHighestGrossingMovieBFCL/irrelevance_54: highest-grossing movie of all time asked; only a movie search (requires a specific title + year, not provided) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 432 OUT54

Tool Calling - Irrelevance: Food, Time & Distance 3 passed1 failed1 skipped

75.0%

IN16 440 OUT662

✓

NoToolForCakeBakingMachineBFCL/irrelevance_218: a baking MACHINE/appliance asked; only a recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 708 OUT138

✓

NoToolForDrivingTravelTimeBFCL/irrelevance_215: driving TRAVEL TIME asked; only a restaurant-bill tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 712 OUT90

×

NoToolForLondonTimezoneNameBFCL/irrelevance_226: the NAME of London's timezone asked; only a local-TIME-by-timezone tool available (it needs the timezone as input and returns a time, not the timezone name) -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2

Failed

IN8 304 OUT275

–

NoToolForPizzaInBostonBFCL/irrelevance_213: a place for pizza asked; a cuisine+location restaurant_finder is available. REVIEW FLAG (fairness): restaurant_finder(cuisine=pizza/Italian, location=Boston) plausibly answers this, so a competent human might legitimately call it. BFCL labels it irrelevant; kept as such and marked Hard because the intended "no call" is debatable. No OTHER query here is a restaurant search, so the tool cannot wrongly satisfy the file's other cases.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'restaurant_finder' (cuisine+location) answers 'good place for pizza in Boston' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForTriangleHypotenuseBFCL/irrelevance_238: the HYPOTENUSE of a triangle asked; only a geo lat/long distance calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 716 OUT159

Tool Calling - Irrelevance: Geo, Travel & Sentiment 6 passed2 failed1 skipped

75.0%

IN119 089 OUT3 159

✓

NoToolForAverageTemperatureBmiBFCL/irrelevance_110: an average December temperature asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 398 OUT66

×

NoToolForCaseLawyerBFCL/irrelevance_95: the lawyer of a case asked; only a case-outcome retriever is available -> no tool fits.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 9

Failed

IN39 923 OUT796

✓

NoToolForCurrentTimeSunriseBFCL/irrelevance_102: the current time asked; only a sunrise/sunset time calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 397 OUT25

✓

NoToolForFamousQuoteBFCL/irrelevance_119: a famous quote asked; only a sentiment-analysis tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 395 OUT32

✓

NoToolForFastestRouteBFCL/irrelevance_114: the fastest route asked; only a closest-airport finder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 399 OUT182

–

NoToolForRestaurantsInLondonBFCL/irrelevance_113: restaurants in London asked; only a generic points-of-interest locator is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'find_pois' (locate points of interest by category) answers 'restaurants in London' (restaurants are POIs) - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForSportsBrainImpactBFCL/irrelevance_120: the neurological impact of sports asked; only a caffeine neurological-impact tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 398 OUT243

✓

NoToolForTrafficConditionBFCL/irrelevance_112: the current traffic condition asked; only an address geocoder is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 397 OUT57

×

NoToolForTravelTimeByCarBFCL/irrelevance_115: travel time by car asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 12

Failed

IN58 782 OUT1 758

Tool Calling - Irrelevance: Legal & Case Lookup 9 passed1 failed

90.0%

IN83 304 OUT1 798

✓

NoToolForAppleCaloriesBFCL/irrelevance_100: calories in an apple asked; only a litigation-cost calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 550 OUT38

✓

NoToolForBrainWeightBFCL/irrelevance_122: average human brain weight asked; only a per-patient neural-activity tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 552 OUT93

✓

NoToolForCaseJudgementBFCL/irrelevance_93: a case judgement asked; only a law-firm impactful-cases lister is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 550 OUT33

✓

NoToolForChineseRestaurantBFCL/irrelevance_97: a Chinese-restaurant recommendation asked; only a lawsuit-filing tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 553 OUT59

✓

NoToolForMovieRecommendationBFCL/irrelevance_116: a movie recommendation asked; only a word-count tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 551 OUT184

×

NoToolForNbaChampionBFCL/irrelevance_118: the 1996 NBA champion asked; only an NBA playoff-brackets display tool is available -> no tool fits.1 AI query 11 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 11

Failed

IN51 330 OUT848

✓

NoToolForPaintingEiffelSettlementBFCL/irrelevance_98: how long to paint the Eiffel Tower asked; only a lawsuit-settlement estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 554 OUT85

✓

NoToolForSecurityGuardSearchBFCL/irrelevance_92: finding a security guard asked; only a lawyer-search tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 560 OUT209

✓

NoToolForSphereVolumeBFCL/irrelevance_104: sphere volume asked; only a circular-park area tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 554 OUT226

✓

NoToolForTrafficLawsBFCL/irrelevance_99: traffic laws asked; only a lawsuit-search tool is available (lawsuits, not laws) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 550 OUT23

Tool Calling - Irrelevance: Math & Science Distractors 8 passed

100.0%

IN25 252 OUT1 157

✓

NoToolForDefiniteIntegralBFCL/irrelevance_9: definite integral asked; only an income-tax calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 159 OUT232

✓

NoToolForIntegralOfCubeBFCL/irrelevance_8: definite integral asked; only a string-to-int converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 152 OUT216

✓

NoToolForLatestMoviesBFCL/irrelevance_15: latest movie releases asked; only a physics final-velocity tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 146 OUT29

✓

NoToolForPerpendicularSlopeBFCL/irrelevance_3: slope of a perpendicular line asked; only a critical-points finder is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 164 OUT60

✓

NoToolForPrimeFactorsBFCL/irrelevance_13: prime factorization asked; only a compound-interest calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 149 OUT55

✓

NoToolForRectangleAreaBFCL/irrelevance_6: rectangle area asked; only a circle-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 160 OUT56

✓

NoToolForTriangleAreaBFCL/irrelevance_0: triangle area asked; only a BMI tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 160 OUT157

✓

NoToolForUpwardAccelerationBFCL/irrelevance_14: acceleration asked (it is just gravity); the tempting max-height and final-velocity tools compute neither the acceleration nor what is asked -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 162 OUT352

Tool Calling - Irrelevance: Math & Statistics 5 passed1 failed

83.3%

IN22 133 OUT2 346

✓

NoToolForBinomialMeanBFCL/irrelevance_67: the MEAN of a binomial experiment asked; only p-value tools are available (they return a p-value, not the mean) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 028 OUT286

✓

NoToolForCivilLawComponentsBFCL/irrelevance_90: the components/categories of Civil Law asked; only a noise-complaint filing tool is available (filing a complaint does not explain law) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 000 OUT320

✓

NoToolForCoinTossMaxHeadsBFCL/irrelevance_63: how many heads CAN be obtained from tossing 3 coins (a count/maximum, not a probability) asked; only a probability-of-an-event calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 004 OUT88

×

NoToolForNullHypothesisStdDevBFCL/irrelevance_68: the STANDARD DEVIATION of a null-hypothesis test asked; only p-value tools are available (they return a p-value, not a standard deviation) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN7 089 OUT1 239

✓

NoToolForPowerOfTwoBFCL/irrelevance_59: raising a number to a power asked; only a linear-equation solver is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 003 OUT107

✓

NoToolForQuadraticEquationBFCL/irrelevance_57: solving a quadratic equation asked; only a genetic-algorithm optimizer and a LINEAR equation solver are available (a quadratic is not linear) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 009 OUT306

Tool Calling - Irrelevance: Medical, Social & Personality 6 passed3 skipped

100.0%

IN20 464 OUT800

✓

NoToolForBigMacCaloriesBFCL/irrelevance_123: calories of a Big Mac asked; only a BMI calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 409 OUT56

✓

NoToolForEconomicHappinessEffectBFCL/irrelevance_126: the effect of economic status on happiness asked; only a happiness-index lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 411 OUT102

✓

NoToolForFriendMbtiBFCL/irrelevance_129: a friend's MBTI asked; only a personality-type evaluator (needs trait percentages) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 409 OUT52

–

NoToolForLatestTechTrendBFCL/irrelevance_124: the latest technology trend asked; only a trending-topics-by-category tool is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_social_trends' (trending topics by category) answers 'latest trend in technology' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForMajorAirportsBFCL/irrelevance_117: major US airports asked; only a coordinate-distance calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 412 OUT184

–

NoToolForMotorNeuronDiseaseBFCL/irrelevance_121: motor neuron disease info asked; only a generic disease-info retriever is available -> no tool fits.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'medical_records_get_disease_info' directly answers 'information on motor neuron diseases' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForPopularBooksBFCL/irrelevance_125: popular books by an author asked; only a recent-tweets retriever is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 411 OUT176

–

NoToolForTwitterMoodBFCL/irrelevance_127: overall twitter mood asked; only a twitter sentiment-analysis tool is available -> BFCL ground truth is no tool call.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'sentiment_analysis_twitter' directly answers 'general mood of twitter regarding the new iPhone' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForVegetableServingsBFCL/irrelevance_128: daily vegetable servings asked; only a personality-test scorer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 412 OUT230

Tool Calling - Irrelevance: Mixed Distractors 8 passed

100.0%

IN26 008 OUT960

✓

NoToolForAirQualityBFCL/irrelevance_40: current air quality asked; only a plant-biomass calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 252 OUT93

✓

NoToolForFavoriteRestaurantBFCL/irrelevance_31: a favorite restaurant recommendation asked; only a cell-division simulator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 248 OUT40

✓

NoToolForGlobalWarmingDocumentaryBFCL/irrelevance_38: a documentary (video) asked; only a scientific-research-paper retriever is available (wrong medium).1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 249 OUT199

✓

NoToolForLasagnaIngredientsBFCL/irrelevance_47: recipe ingredients asked; only a flight-timetable tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 249 OUT237

✓

NoToolForLionDominantTraitBFCL/irrelevance_34: dominant genetic trait of a species asked; only a gene-variant-FREQUENCY lookup is available (different genetic output).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 251 OUT133

✓

NoToolForMultinucleatedCellNameBFCL/irrelevance_30: name of a cell type with multiple nuclei asked; only a blood-cell classifier (by shape/size) is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 255 OUT115

✓

NoToolForPicnicSpotBFCL/irrelevance_37: a picnic spot asked; only a local-fauna information tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 248 OUT122

✓

NoToolForTranslationBFCL/irrelevance_52: natural-language translation asked; only a file-compression tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 256 OUT21

Tool Calling - Irrelevance: Motion, Electricity & Misc 10 passed

100.0%

IN35 428 OUT1 937

✓

NoToolForCarProjectileRangeBFCL/irrelevance_16: projectile-style "how far will a car travel" asked with symbolic v/t/theta; the range tool needs concrete numeric velocity + angle -> nothing to pass -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 553 OUT546

✓

NoToolForConeVolumeBFCL/irrelevance_19: cone volume asked; only an investment-return calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 547 OUT216

✓

NoToolForCurrentClockTimeBFCL/irrelevance_17: current clock time asked; only a travel-time (distance/speed) calculator is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 536 OUT31

✓

NoToolForElectronMassBFCL/irrelevance_24: mass of an electron asked (a knowledge fact); only an electric-current tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 537 OUT25

✓

NoToolForFibonacciOf20BFCL/irrelevance_51: fibonacci number asked; only a cryptocurrency-price tool is available (the fibonacci calculator lives in a different file so it cannot satisfy this query).1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 538 OUT77

✓

NoToolForForceAngleFromMomentumBFCL/irrelevance_18: angle of a force from momentum asked; only a vector-angle-from-XY-components tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 543 OUT325

✓

NoToolForHorrorMovieListBFCL/irrelevance_50: movie recommendations asked; only a Celsius-to-Fahrenheit converter is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 543 OUT288

✓

NoToolForMagneticFieldNearWireBFCL/irrelevance_21: magnetic field near a current-carrying wire asked; only an EM-wave-amplitude tool is available (magnetic-field calculators are deliberately kept in a different file).1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 552 OUT276

✓

NoToolForSineOf45DegreesBFCL/irrelevance_45: sine of an angle asked; only a polygon-drawing tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 538 OUT108

✓

NoToolForTeslaPriceBFCL/irrelevance_46: product price of a car asked; only a currency-exchange-rate tool is available.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 541 OUT45

Tool Calling - Irrelevance: Music Theory & Lookup 8 passed

100.0%

IN25 588 OUT1 118

✓

NoToolForAncientGreekLawmakersBFCL/irrelevance_171: who established laws in Ancient Greece asked (history); only a musical-piece composer search is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 196 OUT324

✓

NoToolForBasketballGameAtVenueBFCL/irrelevance_169: who is playing a basketball game at a venue asked; only concert-locating tools exist.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 199 OUT105

✓

NoToolForBluesScalePatternBFCL/irrelevance_176: the interval pattern of a blues scale asked; only composer lookups are available (get_blues_scale lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 201 OUT168

✓

NoToolForDonQuixoteAuthorBFCL/irrelevance_172: author of the novel Don Quixote asked; only music-composition lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 194 OUT13

✓

NoToolForFamousUsComposersBFCL/irrelevance_170: most famous US composers asked (general knowledge, no specific piece); only a chord-progression creator + piece-specific composer lookups exist -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 197 OUT231

✓

NoToolForGrammyBestAlbumBFCL/irrelevance_177: Grammy Award winner asked; only a song chord-progression lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 202 OUT72

✓

NoToolForPrimaryTriadsOfKeyBFCL/irrelevance_173: primary triads of a key asked; only a genre-common-chords finder is available (primary_triads lives in E2) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 200 OUT121

✓

NoToolForUniversityClassAvailabilityBFCL/irrelevance_168: availability of a university class asked; only concert-availability / concert-search tools are available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 199 OUT84

Tool Calling - Irrelevance: Probability & Legal 5 passed

100.0%

IN14 007 OUT911

✓

NoToolForFaceCardProbabilityBFCL/irrelevance_64: probability of drawing a face card from a deck asked; only a COIN-toss heads probability tool is available (coins, not cards) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 798 OUT164

✓

NoToolForHomeElectricityNeedBFCL/irrelevance_58: home electricity demand (kWh) asked; only a solar-panel COUNT calculator is available (it sizes panels, not electricity need, and needs sunlight hours not given) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN2 801 OUT196

✓

NoToolForNoiseComplaintProcessBFCL/irrelevance_91: how/whether to report a noise complaint asked; only a law-category listing tool is available (it lists legal categories, it does not file or advise on complaints) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 798 OUT96

✓

NoToolForRedMarbleCountBFCL/irrelevance_65: number of red marbles asked; only a determine-population tool is available (it returns a total population from probability + sample size, not a sub-category count) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN2 813 OUT160

✓

NoToolForTenantRightsBFCL/irrelevance_89: explaining tenant rights asked; only a contract-generation tool is available (generating a contract does not explain rights) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN2 797 OUT295

Tool Calling - Irrelevance: Recipes & Cooking 6 passed1 failed1 broken1 skipped

75.0%

IN46 568 OUT63 165

×

NoToolForBestGameCharacterBFCL/irrelevance_206: best game character vs a dragon asked; only a best-WEAPON finder available -> no tool fits.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 5

Failed

IN22 371 OUT535

✓

NoToolForBestMovie2020BFCL/irrelevance_235: best MOVIE of 2020 asked; only a currency-exchange calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 458 OUT124

!

NoToolForBuildingAPcBFCL/irrelevance_209: how to build a PC asked; only a cooking-recipe retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Broken

IN3 453 OUT62 144

✓

NoToolForFamousPlaceInParisBFCL/irrelevance_210: most famous PLACE in Paris asked; only a restaurant-by-dish search available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 455 OUT84

✓

NoToolForGameShortestPathBFCL/irrelevance_208: shortest in-game path asked; only a taxi-fare calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 463 OUT73

✓

NoToolForInchToMeterBFCL/irrelevance_234: LENGTH conversion (inches to metres) asked; only a CURRENCY converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 454 OUT15

✓

NoToolForLowestGameScoreBFCL/irrelevance_207: LOWEST game score asked; only a HIGHEST-score retriever available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 458 OUT98

✓

NoToolForNewYorkPopulationBFCL/irrelevance_225: current POPULATION of New York asked; only a time-zone converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 456 OUT92

–

NoToolForQuickestFlightLondonTokyoBFCL/irrelevance_236: quickest way to fly London->Tokyo asked; a flight-duration tool is available. REVIEW FLAG (fairness): get_flight_duration plausibly answers this ("quickest ... by plane" ~= quickest flight duration), so a competent human might legitimately call it. Kept as BFCL labels it (irrelevant), but this item is borderline. Marked Hard because the intended "no call" is debatable.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_flight_duration' (with flight_type non-stop) plausibly answers 'quickest way ... by plane' - BFCL-mislabeled/borderline-relevant. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Irrelevance: Restaurants & Travel 6 passed2 failed

75.0%

IN48 106 OUT897

×

NoToolForBeijingSunsetBFCL/irrelevance_227: SUNSET time asked; only a SUNRISE calculator available -> no tool fits.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 3

Failed

IN14 371 OUT329

✓

NoToolForClosestItalianRestaurantBFCL/irrelevance_216: FIND the closest Italian restaurant asked; only a currency converter available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 314 OUT50

✓

NoToolForFindSushiRestaurantBFCL/irrelevance_214: FIND a sushi restaurant asked; only a tip calculator available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 316 OUT29

✓

NoToolForNearestPharmacyBFCL/irrelevance_237: FIND the nearest pharmacy asked; only a route-to-a-known-location planner available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 316 OUT81

✓

NoToolForPetFriendlyTrainStationBFCL/irrelevance_230: a pet-friendly TRAIN STATION asked; only hotel search tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 315 OUT52

✓

NoToolForSushiRestaurantsTokyoBFCL/irrelevance_229: FIND sushi RESTAURANTS asked; only hotel-booking tools available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 316 OUT34

✓

NoToolForThaiRestaurantChicagoBFCL/irrelevance_231: FIND a Thai RESTAURANT asked; only a hotel-availability checker available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 317 OUT23

×

NoToolForYogaRetreatBFCL/irrelevance_232: a yoga RETREAT asked; only a HOTEL search tool available. REVIEW FLAG (fairness): a retreat is not a hotel, but hotel_search_find_hotels accepts arbitrary amenities (spa, vegetarian) and a location, so it is somewhat tempting. BFCL labels this irrelevant (retreat != hotel); kept as such and marked Hard because the boundary is borderline.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 3

Failed

IN13 841 OUT299

Tool Calling - Irrelevance: Sports & Games (set 1) 9 passed1 failed

90.0%

IN46 937 OUT2 057

×

NoToolForAssistAverageBFCL/irrelevance_185: assist average asked; only an average-SCORING tool is available -> no tool fits.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 3

Failed

IN14 801 OUT300

✓

NoToolForBestChessMoveBFCL/irrelevance_194: best next move asked; only a board-game-STATISTICS tool is available (the move-suggesting board_analyser lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 574 OUT134

✓

NoToolForChessArrangementCountBFCL/irrelevance_196: total theoretical arrangements asked; only a final-score calculator is available (possible_moves lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 574 OUT239

✓

NoToolForFootballTeamRankingBFCL/irrelevance_186: ranking of a team asked; only a season-MVP tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 570 OUT148

✓

NoToolForIceCreamStoreBFCL/irrelevance_201: find a physical store asked; only a video-game guide is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 567 OUT130

✓

NoToolForMostHeartsInPokerBFCL/irrelevance_199: who holds most of a suit asked; only a poker-dealing tool is available (get_highest_card_holder lives in E4) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 574 OUT188

✓

NoToolForPlayerBiographyBFCL/irrelevance_189: a biographical "who is X" asked; only a top-ranked-athlete lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 567 OUT80

✓

NoToolForTeamRosterBFCL/irrelevance_192: full roster asked; only a top-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 569 OUT77

✓

NoToolForUnoRulesBFCL/irrelevance_198: the rules of a game asked; only a deck-shuffling tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 571 OUT691

✓

NoToolForWorldCupWinnerBFCL/irrelevance_191: who won a tournament asked (no team named); only a per-team match-stats tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 570 OUT70

Tool Calling - Irrelevance: Sports & Games (set 2) 8 passed1 failed

88.9%

IN38 327 OUT957

✓

NoToolForBlackjackAceRuleBFCL/irrelevance_200: a rules question asked; only a poker highest-card-holder tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 469 OUT129

✓

NoToolForBookAuthorBFCL/irrelevance_205: author of a book asked; only a video-game score calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 473 OUT19

✓

NoToolForChessTournamentWinnerBFCL/irrelevance_195: who won a tournament asked; only a board-position move-suggester is available (the game-history fetch_game_stats lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 470 OUT123

✓

NoToolForCreateJigsawBFCL/irrelevance_204: CREATE a puzzle asked; only a puzzle-SOLVING tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 463 OUT187

×

NoToolForHaloPlayerRankBFCL/irrelevance_203: a player's RANK asked; only a player-SCORE lookup is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2

Failed

IN10 584 OUT181

✓

NoToolForLakersTopScorerBFCL/irrelevance_193: top scorer of a team asked; only a team-details tool (roster/results/upcoming, not scorers) is available (sports_team.get_top_scorer lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 468 OUT125

✓

NoToolForMonopolyWinnerBFCL/irrelevance_197: who won a casual game asked; only a possible-moves calculator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 468 OUT32

✓

NoToolForTopTennisPlayerBFCL/irrelevance_190: top ranked PLAYER asked; only a TEAM standing/ranking tool is available (the athlete-ranking get_top_ranked lives in E3) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 467 OUT123

✓

NoToolForWorldSeriesWinnerBFCL/irrelevance_202: who won a real game asked; only a score calculator (from raw points) is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 465 OUT38

Tool Calling - Irrelevance: Sports Statistics 8 passed1 failed

88.9%

IN42 975 OUT1 031

✓

NoToolForBasketballSeasonMvpBFCL/irrelevance_187: who won the MVP asked; only a team-ranking tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 511 OUT66

✓

NoToolForClippersRosterBFCL/irrelevance_179: roster of a team asked; only a two-team game-results tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 504 OUT71

✓

NoToolForCommonChordsInPopBFCL/irrelevance_174: most common chords in a pop song asked; only a key-signature primary-triads tool is available (find_common_chords lives in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 509 OUT247

×

NoToolForCricketMatchParticipantsBFCL/irrelevance_180: who is playing in cricket matches today asked; only a cricket-schedule tool (dates/matches, not participants) is available -> no tool fits.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 3

Failed

IN14 905 OUT401

✓

NoToolForLaLigaPlayersBFCL/irrelevance_181: who played in a whole league asked; only a two-named-team last-match-result tool exists.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 504 OUT68

✓

NoToolForMoonlightSonataComposerBFCL/irrelevance_175: composer of Moonlight Sonata asked; only a blues-scale generator is available here (all composer-lookup tools are isolated in E1) -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 507 OUT39

✓

NoToolForNbaMvpWinnerBFCL/irrelevance_184: who won an MVP award asked (no player named); only a per-player stats tool that requires a player name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 513 OUT58

✓

NoToolForTopAssistPlayerBFCL/irrelevance_178: top ASSIST player asked; only a top-GOAL-scorer tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 508 OUT50

✓

NoToolForWimbledonChampionBFCL/irrelevance_183: winner of a tournament asked (no person named); only a celebrity-info tool that requires a name is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 514 OUT31

Tool Calling - Irrelevance: Time, Weather & Misc 2 passed1 failed1 skipped

66.7%

IN10 350 OUT467

×

NoToolForCityDistanceKmBFCL/irrelevance_239: DISTANCE (km) between two cities asked; only a get_date tool that returns the TIME difference between two locations is available (its 'unit' arg mentions distance, but it computes time) -> no tool fits.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 1

Failed

IN5 192 OUT223

–

NoToolForCurrentTimeSydneyBFCL/irrelevance_228: current time in Sydney asked; a local-time-by-city tool is available. REVIEW FLAG (fairness): get_local_time(location='Sydney') directly answers this, so a competent human would legitimately call it. BFCL nonetheless files it under irrelevance; kept as such to preserve the source label, but this item is effectively RELEVANT and should be reviewed. Marked Hard because the "no call" expectation is contradicted by the tool itself.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Not a valid irrelevance item (item flaw): the case's own tool 'get_local_time' directly answers 'current time in Sydney' - BFCL-mislabeled relevant. Kept for traceability.

Skipped

IN0 OUT0

✓

NoToolForTomatoCaloriesBFCL/irrelevance_222: CALORIES in a tomato asked; only a grocery-item-details lookup available. REVIEW FLAG (fairness): item_details at 'detailed' level could arguably surface nutrition info, and it requires a store_location the user never gave; BFCL labels it irrelevant (it is a grocery-store item lookup, not a nutrition-facts service). Kept as such; marked Medium as borderline.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN2 580 OUT44

✓

NoToolForWriteABookBFCL/irrelevance_217: "can you write a book" asked; only a recipe-CREATION tool available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN2 578 OUT200

Tool Calling - Irrelevance: Weather, Environment & Services 8 passed2 failed

80.0%

IN130 155 OUT3 064

✓

NoToolForBestCo2ReductionBFCL/irrelevance_108: the best way to reduce CO2 asked; only a numeric emission-reduction estimator is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 662 OUT413

✓

NoToolForBestMonthToVisitBFCL/irrelevance_101: best month to visit Hawaii asked; only an average-monthly-temperature tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 660 OUT126

×

NoToolForCactusNutrientsBFCL/irrelevance_109: a plant's nutrient needs asked; only a weekly watering-needs calculator is available -> no tool fits.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 2

Failed

IN11 202 OUT262

✓

NoToolForCurrentTimeWeatherBFCL/irrelevance_103: the current time asked; only a current-weather forecast tool is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 661 OUT40

✓

NoToolForGardenSpeciesCompositionBFCL/irrelevance_107: garden species composition asked; only a soil sand/silt/clay composition analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN3 664 OUT49

×

NoToolForGmoYogaMatBFCL/irrelevance_111: a GMO yoga mat asked; only an eco-friendly product locator is available (GMO is not eco-friendly) -> no tool fits.1 AI query 17 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 0 Values: response.ToolCalls.Count = 17

Failed

IN89 646 OUT1 622

✓

NoToolForImpactfulCasesBFCL/irrelevance_94: a firm's impactful cases asked; only a single-case detail lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 671 OUT79

✓

NoToolForPaintingEiffelCaseResultBFCL/irrelevance_96: how long to paint the Eiffel Tower asked; only a law-case result lookup is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 663 OUT87

✓

NoToolForPollinationFlowersBFCL/irrelevance_105: top pollination flower species asked; only a route elevation-profile plotter is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Easysource:BFCL

Passed

IN3 665 OUT254

✓

NoToolForTomatoFertilizerBFCL/irrelevance_106: best fertilizer for tomatoes asked; only a soil-type analyzer is available -> no tool fits.1 AI query 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 661 OUT132

Tool Calling - Parallel Calls: Art, Music & Local Services 1 passed9 failed1 skipped

10.0%

IN231 818 OUT8 503

–

CalculateDrivingDistancesShortestRouteBFCL/parallel_38: two calc_distance calls — New York→Washington DC and Los Angeles→San Francisco. Tightened per methodology: the prompt explicitly enables "shortest route", so shortest_route == true is required (schema default is false).0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Skipped: Ambiguous scope (item flaw, not model behavior): the trailing 'with optional parameter shortest route enabled' can reasonably apply to only the second leg (its nearest clause) or to both; the BFCL ground truth requires it on both, but a competent human would not answer deterministically. Kept for traceability.

Skipped

IN0 OUT0

×

CheckInstrumentAvailabilityInTwoCitiesBFCL/parallel_42: two check_instrument_availability calls — Yamaha P125 in Berlin and Madrid.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN16 045 OUT571

×

FindMusicStoresInTwoCitiesBFCL/parallel_41: two find_music_instrument_store calls — San Francisco and Chicago; instruments {Yamaha acoustic guitar, Kawai piano} in any order and either casing.1 AI query 28 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 28 expectedCount = 2

Failed

IN87 495 OUT2 699

✓

FindRockAndJazzConcertsInTwoCitiesBFCL/parallel_43: four concert_finder calls — {San Francisco, New York} x {rock, jazz} over the next month. time_period may be omitted (defaults to 30 days ≈ "next month"). Isolated from concert_find_nearby (parallel_44) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN11 591 OUT435

×

GetExhibitionDetailsForTwoMuseumsBFCL/parallel_40: two museum_exhibition_detail calls — 'Wonder of Nature' at the Louvre and 'Age of Reptiles' at the British Museum. visitor_type {child, adult} (unordered). Tightened per methodology: the prompt explicitly asks for cost "for children and adult", so both visitor types are required (schema default is ['adult'] only).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN16 297 OUT826

×

GetMostPlayedPopAndRockSongsBFCL/parallel_45: two musicCharts_getMostPlayed calls — most played Pop and Rock song in Australia. Optional duration omitted (ground truth accepts the schema default of 0).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 2

Failed

IN11 425 OUT244

×

GetMuseumInfoForTwoCitiesBFCL/parallel_39: two museum_info_get_info calls — Washington D.C. and Paris; details {Opening hours, Adult tickets, Child tickets} in any order.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN20 880 OUT879

×

GetRgbAndHexForThreeColorsBFCL/parallel_37: three color_converter_get_color_info calls — navy, purple, maroon; each with {RGB, HEX} (unordered).1 AI query 15 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 15 expectedCount = 3

Failed

IN27 998 OUT1 086

×

GetTeamRankingsInTwoLeaguesBFCL/parallel_47: two get_team_ranking calls — Barcelona in UEFA Champions League and Manchester United in La Liga.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN11 573 OUT309

×

MixPaintColorsForTwoRequestsBFCL/parallel_36: two paint_color_mixture calls — watercolor magenta and acrylic navy.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN16 065 OUT743

×

WinningPercentageForTwoTeamsTwoSeasonsBFCL/parallel_46: four calculate_winning_percentage calls — {Lakers, Bulls} x {2018, 2020}.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN12 449 OUT711

Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 1) 3 passed10 failed

23.1%

IN847 331 OUT23 695

×

AverageFourNumberSetsBFCL/parallel_132: four calculate_average calls — one per list of numbers (element order preserved).1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 4

Failed

IN26 095 OUT1 257

×

CalculateBmiFourPeopleMetricBFCL/parallel_134: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt states "All measurements are in the metric system", so unit == "metric" is required (also the default).1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 4

Failed

IN18 959 OUT950

×

CalculateCityDistancesInKilometersBFCL/parallel_135: three geo_distance_calculate calls — NY->LA->Miami->NY, in kilometers. Tightened: the tool defaults to miles but the prompt asks "in kilometers", so units == "kilometers" is REQUIRED (omission would return miles, contradicting the request — the ground truth's "omit" option is unsafe here).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 3

Failed

IN23 367 OUT613

✓

CalculateGpsDistancesAlongRouteBFCL/parallel_133: three calculate_distance calls over GPS coordinate pairs (Paris->Rome->Athens-> Giza), each in kilometers. Uses the coordinate-based tool (coord1/coord2), not the city-name distance tools that are also visible.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 905 OUT3 320

✓

CompareAverageTemperatureNyAndLaBFCL/parallel_128: two average_temperature calls — New York 7 days in Fahrenheit, Los Angeles 7 days in Celsius. Tightened: the prompt explicitly requests Fahrenheit for NY, so temp_unit == "Fahrenheit" is required (also the default); Celsius is non-default and required for LA.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 362 OUT192

✓

CreateTwoHistogramsWithFiveBinsBFCL/parallel_129: two create_histogram calls — one per data set, 5 bins each (data order preserved).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 826 OUT467

×

EstimateElephantPopulationThreeScenariosBFCL/parallel_125: three elephant_population_estimate calls — current 500 at 2%, 1.5%, 2.5% over 10y.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3

Failed

IN25 116 OUT1 506

×

FindItalianRestaurantsInNyAndLaBFCL/parallel_130: two find_restaurants calls — New York and Los Angeles, Italian, 4 results each, dietary_requirements [vegan, gluten-free]. Distinct from restaurant_find_nearby (this tool carries food_type + number, which the prompt supplies, so it is the correct choice here).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN23 375 OUT1 310

×

FindRestaurantsByDietInThreeCitiesBFCL/parallel_127: three restaurant_find_nearby calls — New York, Los Angeles, Chicago, each with dietary_preference [Vegan, Gluten-free, Dairy-free]. List order follows the prompt (matches ground truth).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN13 511 OUT834

×

PredictEvolutionDarwinAndLamarckBFCL/parallel_126: two prediction_evolution calls — African Elephant over 5000y, Darwin then Lamarck. Tightened per methodology: the prompt explicitly says "using the Darwin model", so model == "Darwin" is required for the first call (ground truth also accepts omission via the 'Darwin' default).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN17 879 OUT883

×

ProjectPopulationGrowthJapanAndIndiaBFCL/parallel_124: four population_projections calls — Japan/10y and India/20y, each at the current (default) growth rate and at an explicit rate. "current growth rate" maps to the default (growth_rate omitted, accepts null or 0.01); the explicit 1.5%/2.1% rates are required.1 AI query 16 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 4

Failed

IN26 025 OUT1 428

×

RouteThreeLegsAvoidingTollsBFCL/parallel_131: three map_routing_fastest_route calls — SF->Palo Alto->San Jose->home, all avoiding tolls. Tightened: the prompt explicitly says "avoiding toll roads", so avoid_tolls == true is required for every leg (non-default value stated in the prompt).1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN18 843 OUT811

×

ShortestCityDistanceBusWithAndWithoutTransferBFCL/parallel_136: two city_distance_find_shortest calls — NY->LA by bus, first without transfer then allowing transfer. Tightened: the prompt explicitly says "using a bus", so transportation == "bus" is required for both (also the default). allow_transfer: false/omitted for the first, true for the second.1 AI query 100 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN628 068 OUT10 124

Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 2) 5 passed9 failed

35.7%

IN209 392 OUT10 239

✓

CalculateAreaOfThreeTrianglesBFCL/parallel_143: three calc_area_triangle calls — (10,5), (15,7), (20,10).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 962 OUT456

×

CalculateBinomialProbabilityBasketballBFCL/parallel_161: three calculate_binomial_probability calls — basketball shooter at 60%: (10,7), (15,10), (20,15). probability_of_success == 0.6 is required (prompt states 60%). ISOLATED here (away from prob_dist.binomial in ToolCalling-Parallel-D3): both tools compute binomial probability, and this prompt does not name a function, so having both visible would make the choice genuinely ambiguous. Kept apart so exactly one correct tool is available.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN14 210 OUT1 467

×

CalculateBmiInMetersForThreePeopleBFCL/parallel_138: three calculate_BMI calls (weight kg, height meters) — John, Sarah, Mike.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3

Failed

IN25 661 OUT1 006

×

CalculateCelestialDistancesInMilesBFCL/parallel_149: two calculate_distance calls between celestial bodies (Mars-Venus, Mars-Jupiter), in miles. Uses the body-name distance tool (body1/body2). Tightened: prompt asks "in miles", so unit == "miles" is required (tool default is kilometers, so omission would be wrong).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN18 625 OUT976

×

CalculateClockAnglesAtThreeTimesBFCL/parallel_145: three calculate_clock_angle calls — 3:15, 8:20, 11:50. Tightened: the prompt asks for results "rounded to 2 decimal places", so round_to == 2 is required (also the default).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN19 555 OUT1 219

✓

CalculateFactorialOfFourNumbersBFCL/parallel_144: four math_factorial calls — 5, 3, 4, 2.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 972 OUT225

×

CalculateLightTravelTimeThreeLegsBFCL/parallel_147: three light_travel_time calls — 4.22, 6.1, 5.88 light years. Tightened: the prompt explicitly states the speed of light (299792458 m/s), so speed_of_light == 299792458 is required (also default).1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 3

Failed

IN24 949 OUT1 133

×

CalculateSpeedForCarBicycleTrainBFCL/parallel_148: three calculate_speed calls — car 500m/25s in km/h, bicycle 1000m/200s in m/s, train 10000m/600s in km/h. Each output unit is explicitly named in the prompt, so to_unit is required.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN19 809 OUT903

×

FetchAllDataFieldsForTwoEmployeesBFCL/parallel_139: two employee_fetch_data calls — Tech Solutions, employees 12345 and 67890, each fetching all four data fields (order follows the prompt / ground truth).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 775 OUT443

×

FindDiCaprioDramaAndComedyMoviesBFCL/parallel_140: two imdb_find_movies_by_actor calls — Leonardo DiCaprio, Drama in 2010 and Comedy in 2012. Tightened: the prompt explicitly names each category, so category is required for both.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 559 OUT458

✓

GetImaxAndTwoDTheaterReleasesBFCL/parallel_141: two get_theater_movie_releases calls — New York IMAX over 7 days, Los Angeles 2D over 14 days. Tightened: the prompt explicitly names each format, so format is required for both.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 793 OUT372

×

PlotTwoSineWavesBFCL/parallel_146: two plot_sine_wave calls with all five parameters supplied by the prompt.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 790 OUT452

✓

SortFourListsAscendingAndDescendingBFCL/parallel_137: eight array_sort calls — four lists, each sorted ascending AND descending (list element order preserved as given).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN9 848 OUT871

✓

UpdateTwoCustomersNameAndEmailBFCL/parallel_142: two update_user_info calls — users 12345 and 67890, both set name "John" and email "example@.com". update_info is a nested object. Tightened: the prompt names the 'CustomerInfo' database, so database == "CustomerInfo" is required (also the default).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 884 OUT258

Tool Calling - Parallel Calls: Data, Geo & Life Sciences (set 3) 3 passed8 failed

27.3%

IN178 424 OUT7 886

×

AreaUnderCurveForTwoPolynomialsBFCL/parallel_150: two mathematics_calculate_area_under_curve calls — polynomial [3,-2,1] over [-1,2] and [1,0,-1] over [0,3] (coefficient/limit order preserved).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 149 OUT454

×

BinomialProbabilitiesThreeGamesBFCL/parallel_160: three prob_dist_binomial calls — (20,5), (50,15), (100,30), each with p == 0.3 (prompt states a 0.3 win probability). The prompt explicitly names 'prob_dist.binomial'; the colliding calculate_binomial_probability tool is kept in a separate script (ToolCalling-Parallel-D2).1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3

Failed

IN23 311 OUT1 355

✓

CalculateBmiSystemFourPeopleBFCL/parallel_154: four calculate_bmi calls (weight kg, height cm). Tightened: the prompt says "use the metric system for all calculations", so system == "metric" is required (also the default).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 132 OUT517

×

DiceRollProbabilitiesThreeScenariosBFCL/parallel_159: three probability_dice_roll calls — roll 4 three times and roll 2 twice on a six-sided die (die_sides default 6, accepts omission), then roll 7 twice on an eight-sided die (die_sides == 8 required, non-default).1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN17 637 OUT992

✓

GenerateNormalVariatesTwoConfigsBFCL/parallel_158: four random_normalvariate calls — two at (mu 5, sigma 2) and two at (mu 10, sigma 3). Duplicate calls: each configuration must appear exactly twice (multiplicity 2).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 068 OUT504

×

PredictPricesForFourHousesBFCL/parallel_157: four predict_house_price calls — New York (3/2/1500), Los Angeles (4/3/2000), Chicago (2/1/1200), Miami (3/2/1800).1 AI query 16 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 4

Failed

IN25 370 OUT1 473

✓

RaiseTwoNumbersToPowersBFCL/parallel_152: two math_power calls — 3^5 and 2^3, no modulus (mod omitted / null).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 688 OUT139

×

RunLinearRegressionWithAndWithoutStandardizationBFCL/parallel_155: two run_linear_regression calls — same predictors [Age, Income, Education] and target "Spending Score", first without standardization then with. standardize is explicitly requested (false then true), so it is required for both.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN16 881 OUT566

×

TrainRandomForestFourConfigsBFCL/parallel_156: four random_forest_train calls on "data_random_forest" — (100,10), (200,20), (300,30), (400,40). The prompt explicitly names this function, disambiguating it from train_random_forest_classifier visible in this script.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 4

Failed

IN13 038 OUT545

×

TrainRfClassifierTwoConfigsBFCL/parallel_153: two train_random_forest_classifier calls — dataset1 (depth 10, 100 trees) and dataset2 (depth 20, 200 trees). The prompt explicitly names this function, disambiguating it from the other random-forest training tool (random_forest_train) visible in this script.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN16 869 OUT504

×

TriangleAreasInDifferentUnitsBFCL/parallel_151: three geometry_area_triangle calls in different units — (15,20) square meters, (25,30) square feet, (35,40) square inches. First unit is the default (accepts omission / square meters); feet and inches are non-default and required.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN29 281 OUT837

Tool Calling - Parallel Calls: Finance, Stats & Legal 2 passed11 failed

15.4%

IN237 380 OUT9 315

✓

BuildTwoDecisionTreeClassifiersBFCL/parallel_12: two model_DecisionTreeClassifier calls — gini/depth 5 and entropy/depth 10, both random_state 1.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 444 OUT251

×

CalculateFutureValueForThreeTermsBFCL/parallel_18: three financials_calculate_future_value calls — years 1, 5, 10 (same 1000 @ 0.05).1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 3

Failed

IN35 899 OUT653

×

CalculatePresentValueForThreeTermsBFCL/parallel_14: three calculate_present_value calls — years 10, 20, 30 (same payment 1000, rate 0.05).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN18 955 OUT735

×

CapitalGainsTaxForTwoStatesBFCL/parallel_15: two calculate_capital_gains_tax calls — CA (15000/25000) and FL (20000/50000).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 2

Failed

IN13 689 OUT529

×

ComputePythagorasForTwoTrianglesBFCL/parallel_10: two math_pythagoras calls — sides (3,4) and (5,12).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 932 OUT438

×

ConfidenceIntervalForTwoSampleSizesBFCL/parallel_13: two confidence_interval_calculate calls — sample sizes 50 and 150. Tightened per methodology: the prompt explicitly requests "95% confidence interval", so confidence_level == 0.95 is required (schema default is 0.9, but the prompt states the value).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN23 565 OUT1 133

✓

CourtCaseStatusAndTrialDatesBFCL/parallel_22: four court_info_get_case_status calls — cases XY1234 (LA) and GH5678 (Orange County), each queried for status (details may be omitted via the 'status' default) and for trial_date.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 758 OUT1 109

×

GetStockPriceAndVolumeForThreeCompaniesBFCL/parallel_17: three get_stock_data calls — AAPL, GOOG/GOOGL, MSFT; each with data_points {price, volume} (unordered).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN13 397 OUT479

×

LoanEligibilityForTwoInstitutionsBFCL/parallel_20: two loan_eligibility_check calls — HSBC (500k/100k) and Wells Fargo (700k/120k).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN18 203 OUT642

×

MortgagePaymentForThreeLoanTermsBFCL/parallel_19: three calculate_mortgage_payment calls — loan terms 15, 20, 30 (400000 @ 0.04).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN19 059 OUT758

×

PredictHousePriceInTwoCitiesBFCL/parallel_11: two ml_predict_house_price calls — New York 3000 sq ft, Los Angeles 4000 sq ft.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN17 853 OUT531

×

ReturnOnInvestmentGainAndLossBFCL/parallel_16: two calculate_return_on_investment calls — gain 500 on 2000, loss -1000 on 5000.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2

Failed

IN28 687 OUT940

×

SearchMoneyLaunderingConvictionsBFCL/parallel_21: two law_crimes_search calls — money laundering in San Francisco 2019 and Texas 2018.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN17 939 OUT1 117

Tool Calling - Parallel Calls: Legal, History & Environment 1 passed11 failed1 broken1 skipped

7.7%

IN1 338 732 OUT87 959

✓

BookTwoConnectingFlightsBFCL/parallel_30: two book_flight calls — San Francisco→Tokyo (2022-05-03) and Tokyo→Sydney (2022-05-18).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 061 OUT197

×

CalculateAlimonyForTwoDurationsBFCL/parallel_23: two alimony_calculator_ca_calculate calls — durations 10 and 20 years (10000/3000 income).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN19 409 OUT1 176

×

CalculateWasteForFamilyAndBachelorBFCL/parallel_29: two waste_calculation_calculate calls — LA family (2 adults, 2 children) and New York bachelor (1 single). The population argument is a nested object.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN19 373 OUT992

×

FetchTwoHistoricalFactsBFCL/parallel_31: two history_fact_fetch calls — Treaty of Paris and Magna Carta. Optional depth/year omitted (ground truth accepts the schema defaults).1 AI query 26 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 26 expectedCount = 2

Failed

IN86 830 OUT1 782

×

FindClassicalConcertsWithParkingBFCL/parallel_44: two concert_find_nearby calls — classical concerts near Berlin and Paris next Friday. Tightened per methodology: the prompt explicitly asks for concerts "with available parking", so amenities == ['Parking'] is required (this also matches the schema default, but the prompt states it). Isolated from concert_finder (parallel_43) into a separate script: both locate concerts by location+genre and would otherwise create an ambiguous tool choice.1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2

Failed

IN30 826 OUT956

–

FindCourtCasesByNumberAndTypeBFCL/parallel_26: four court_case_find calls — the cross product of case numbers {67813, 71249} and types {Civil, Criminal}; case_number is a single-element array; Civil may be omitted via its default. FLAGGED as borderline (see report): the prompt "case numbers '67813','71249' for type 'Civil' and 'Criminal'" is ambiguous about the pairing; BFCL ground truth expects all four combinations, kept faithful.0 AI queries 0 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Skipped: Ambiguous call decomposition (item flaw, not model behavior): 'case numbers 67813, 71249 for type Civil and Criminal' has no single deterministic reading - two calls (both numbers per type, case_number as a list) is as valid as the BFCL 2x2 = four single-number calls; a competent human would not answer deterministically. Kept for traceability.

Skipped

IN0 OUT0

×

FindNatureReservesNearTwoCitiesBFCL/parallel_27: two nature_reserve_find_nearby calls — Berkeley (10km, picnic tables + restrooms) and Tokyo (5km, playgrounds + biking trails). Amenity list order is unordered.1 AI query 99 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN650 478 OUT10 466

×

GetCaseDetailsWithHistoryAndLitigantsBFCL/parallel_24: two law_case_get_details calls — cases 28473 and 64725. Tightened per methodology: the prompt explicitly asks for "their history and details of litigants", so include_history == true and include_litigants == true are required (schema defaults are false).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 961 OUT452

!

GetEventsForTwoPresidenciesBFCL/parallel_32: two us_history_events_by_presidency calls — Abraham Lincoln and George Washington. Optional start_year/end_year omitted (ground truth accepts the schema defaults).1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Broken

IN38 439 OUT63 303

×

GetPresidentsAndVicePresidentsByYearBFCL/parallel_33: four get_president_and_vp calls — president for 1980 and 2016, vice president for 1975 and 2011.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4

Failed

IN20 867 OUT1 105

×

GetReligionInfoForTwoEmpiresBFCL/parallel_35: two ancient_empires_get_religion_info calls — Persian Empire and Mauryan Empire. Tightened per methodology: the prompt explicitly asks for "religious history and influences", so include_influences == true is required (schema default is false).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 895 OUT682

×

GetTemperatureForTwoCitiesBFCL/parallel_28: two get_current_and_future_temperature calls — Seattle and Los Angeles, next 3 hours.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 835 OUT254

×

LookupLawsuitsForTwoYearsBFCL/parallel_25: two lawsuit_lookup calls — Dara Inc for years 2019 and 2018.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 50 expectedCount = 2

Failed

IN395 839 OUT4 361

×

TrackChristianityInTwoRegionsBFCL/parallel_34: two religion_history_track calls — Christianity in Egypt and Turkey, 100–1500 A.D.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 11 expectedCount = 2

Failed

IN25 919 OUT2 233

Tool Calling - Parallel Calls: Markets & Environment 3 passed10 failed

23.1%

IN675 554 OUT18 652

×

CalculateCagrForTwoInvestmentsBFCL/parallel_176: two calculate_cagr calls — (5000,7000,5) and (8000,12000,3).1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 2

Failed

IN48 643 OUT1 664

×

CalculateEmissionSavingsForSolarAndWindBFCL/parallel_186: two calculate_emission_savings calls — solar(12 months) and wind(8 months), Midwest region. Prompt states the Midwest region explicitly -> region required (accept "Midwest"/"Midwest region").1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN23 535 OUT779

×

CompareFutureValueWithAndWithoutContributionsBFCL/parallel_182: two finance_calculate_future_value calls — (5000,0.07,10, no contribution) and (3000,0.06,10, contribution 200). First scenario states "no additional contributions" -> null or 0.1 AI query 69 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 69 expectedCount = 2

Failed

IN414 595 OUT7 590

✓

ComputeEuclideanNormForThreePointsBFCL/parallel_183: three math_hypot calls — (5,7), (10,15), (20,25). z omitted or 0 (2D points).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 150 OUT344

×

EstimatePopulationsForTwoSpeciesAcrossTwoYearsBFCL/parallel_185: four estimate_population calls — Bengal Tigers/India and African Elephants/Kenya for 2021 and for 2023. NOTE: upstream ground truth's third entry lists year "" (omitted -> default 2020), which is an upstream error: the prompt explicitly asks for 2023 for "both these species". We tighten both 2023 estimates to year == 2023 (fair deterministic answer).1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4

Failed

IN20 135 OUT1 178

×

FindRootsForTwoQuadraticEquationsBFCL/parallel_184: two algebra_quadratic_roots calls — (3,7,2) and (5,-4,1).1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2

Failed

IN29 422 OUT1 229

×

GetClosingPricesForTwoCompaniesOnTwoDatesBFCL/parallel_178: four get_stock_price (by-date variant) calls — the 2x2 cross product of {Microsoft, Apple} x {2022-01-01, 2022-02-01} on NASDAQ. Upstream ground truth's first entry loosely lists company_name as [Microsoft, Apple]; the deterministic human answer is the four distinct (company, date) combinations, which we require. exchange NASDAQ (default) may be omitted.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN14 231 OUT885

×

GetDetailedAirQualityForThreeDaysBFCL/parallel_187: three get_air_quality calls — New York City, detail true, for 2023-05-05, 05-04, 05-03. Prompt asks for PM2.5/PM10/ozone/sources -> detail == true; "today/yesterday/day before" with today = May 5, 2023.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3

Failed

IN25 262 OUT1 000

✓

GetFourPriceTypesForTwoCompaniesBFCL/parallel_180: eight stock_price calls — {Microsoft, Apple} x {Open, Close, High, Low}, days 30. Prompt explicitly lists all four data types, so data_type must be set explicitly (required) to distinguish the eight calls; omission (default 'Close') cannot represent the four requested types.1 AI query 8 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Passed

IN9 442 OUT625

×

GetPricePerOunceForFourMetalsBFCL/parallel_177: four get_metal_price calls — gold, silver, platinum, palladium, measure 'ounce'. Prompt asks "price per ounce" -> measure == "ounce" required.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4

Failed

IN19 733 OUT776

×

GetStockPricesForFourCompaniesAndDurationsBFCL/parallel_181: four get_stock_prices calls — Apple(1 week), Microsoft(2 weeks), Amazon(3 weeks), Tesla(1 month). Each 'companies' argument is a single-element list.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN28 930 OUT582

✓

GetTrafficInfoForThreeLegsWithModesBFCL/parallel_188: three get_traffic_info calls — NY->LA (driving), LA->SF (bicycling), SF->NY (transit). First leg explicitly says "driving" -> mode == "driving" required (tightened; also the default).1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 763 OUT631

×

ProjectFutureValueForThreeStocksBFCL/parallel_175: three portfolio_future_value calls — AAPL(5000,0.07,5), MSFT(8000,0.06,7), AMZN(10000,0.08,10).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN19 713 OUT1 369

Tool Calling - Parallel Calls: Mathematics & Physics (set 1) 5 passed8 failed

38.5%

IN713 810 OUT13 441

×

AreaOfTwoRightTrianglesInMetersBFCL/parallel_68: two calculate_area calls — right triangles base/height 12x15 and 18x24 in meters. unit required (tool default is 'cm'; the prompt states meters/square meters).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 2

Failed

IN12 222 OUT480

×

CheckPlayerStatusAcrossThreeSeasonsBFCL/parallel_49: three player_status_check calls — player 3142 in RocketLeague for seasons 2017/2018/2019. Seasons are explicitly given, so each is required (default 2023 not applicable).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN17 875 OUT829

✓

CircumferenceOfFourCirclesInCmBFCL/parallel_65: four calculate_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt gives every radius in cm (e.g. "5cm") and the tool default unit is 'm', so a correct call must pass unit == 'cm'/'centimeter' (ground truth requires it on the first call and merely allows omission on the rest; we require it consistently since it is stated).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 217 OUT591

×

ConvertEuroAndPoundsToDollarsBFCL/parallel_59: two get_conversion_cost calls — 15000 Euro->dollars and 200 pounds->dollars.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN16 897 OUT684

×

FinalVelocityForFourDropsBFCL/parallel_81: four calculate_final_velocity calls — heights 10/20/15/25, initial_velocity 0 (stated), gravity 9.81 (default, may be omitted).1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 4

Failed

IN13 632 OUT382

✓

FindItalianAndJapaneseRestaurantsBFCL/parallel_52: two restaurant_finder calls — Italian in New York (preferences ['Vegetarian']) and Japanese in Los Angeles (delivery; ['Delivery'] or omitted since default is ['Delivery']).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 888 OUT239

×

FindRootsOfTwoQuadraticsAlgebraBFCL/parallel_62: two algebra_quadratic_roots calls — 3x^2+4x+2 and 5x^2-7x+3.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 461 OUT922

×

FindSoupAndVeganSaladRecipesBFCL/parallel_51: two recipe_search_find calls — Chicken Noodle Soup (diet omitted/default 'Keto') and a Vegan Salad (diet == 'Vegan', dish name accepts several phrasings).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN16 649 OUT442

×

GcdOfJohnAndMaryNumbersBFCL/parallel_80: two math_gcd calls — gcd(36,48) and gcd(60,96). Same tool as parallel_76.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 2

Failed

IN26 793 OUT624

✓

GcdOfTwoPairsBFCL/parallel_76: two math_gcd calls — gcd(45,60) and gcd(81,27).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 882 OUT146

×

GetPokemonMovesAndCheckSolarBeamBFCL/parallel_48: two PokemonGO_get_moves calls — Pikachu's move list (move omitted/default 'Run') and a check whether Bulbasaur can learn 'Solar Beam'.1 AI query 100 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN557 280 OUT6 878

✓

PrimeFactorsFormattedAndArrayBFCL/parallel_74: four get_prime_factors calls — numbers 4567 and 7890, each formatted true and false (user wants "a formatted string as well as an array").1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 164 OUT1 086

✓

SaveGameProgressEasyAndHardBFCL/parallel_50: two game_save_progress calls — stage 7 easy and stage 3 hard; level omitted/default 'user'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 850 OUT138

Tool Calling - Parallel Calls: Mathematics & Physics (set 2) 3 passed10 failed

23.1%

IN369 596 OUT12 937

×

AreaOfThreeCirclesInMetersBFCL/parallel_66: three geometry_area_circle calls — radii 5/10/15 in meters (default is meters, so units may be omitted).1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 18 expectedCount = 3

Failed

IN36 168 OUT1 202

✓

BookTwoHotelRoomsBFCL/parallel_57: two hotel_booking_book calls — Sheraton (NY, 2 adults/1 child) and Marriott (LA, 1 adult/2 children).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 387 OUT364

×

CheckAvailabilityForTwoHotelsBFCL/parallel_56: two hotel_booking_check_availability calls — Queens Hotel (Berlin) and Royal Hotel (Paris), each with check-in/out dates (several date formats accepted).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN18 051 OUT719

×

CircumferenceOfFourCirclesInMetersBFCL/parallel_70: four geometry_circumference calls — radii 5/10/15/20. Tightened per methodology: the prompt asks for output in meters and gives radii in m, but the tool default unit is 'cm', so a correct call must pass units == 'm'/'meters' (ground truth requires it on the first call and only allows omission on the rest; we require it consistently since it is explicitly stated).1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 5 expectedCount = 4

Failed

IN13 140 OUT478

✓

DerivativeOfPolynomialAndItsDerivativeBFCL/parallel_71: two calculate_derivative calls — 3x^3-2x^2+5x-7 at x=4, then its derivative 9x^2-4x+5 at x=2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN12 284 OUT408

✓

FinalVelocityForThreeDragRaceRoundsBFCL/parallel_83: three final_velocity calls — (v0=0,a=5,t=10), (v0=10,a=7,t=8), (v0=20,a=4,t=12).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 516 OUT530

×

FindGroceryStoresByCriteriaBFCL/parallel_55: two grocery_store_find_by_criteria calls — NYC ['24 hours'] and San Diego ['Home Delivery'].1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2

Failed

IN27 712 OUT1 028

×

GetTwoCookingRecipesBFCL/parallel_53: two get_cooking_recipe calls — Lasagne Bolognese for 4 and Caesar Salad for 2.1 AI query 42 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 42 expectedCount = 2

Failed

IN154 678 OUT3 961

×

GetTwoExchangeRatesBFCL/parallel_58: two get_exchange_rate calls — USD->AUD and USD->CAD.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN17 269 OUT388

×

HighestCommonFactorOfFourPairsBFCL/parallel_77: four math_hcf calls — (45,60), (90,120), (36,48), (72,96). FLAGGED (see final report): the prompt is phrased as chaining ("use that result to find the HCF with...") but the ground truth treats it as four INDEPENDENT parallel calls on the literal pairs; a human might interpret it as sequential composition. Kept BFCL-faithful (four independent literal-pair calls) since parallel semantics preclude feeding one call's result into another.1 AI query 7 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 4

Failed

IN28 824 OUT1 543

×

OrderSameItemsFromTwoWholeFoodsLocationsBFCL/parallel_54: two whole_foods_order calls — same two items (order-independent) at downtown and uptown, size large.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 2

Failed

IN14 410 OUT238

×

PrimeFactorizeAsDictionaryBFCL/parallel_79: two prime_factorize calls — 4567 and 7890, return_type 'dictionary' (explicitly requested).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN17 369 OUT1 217

×

SolveTwoQuadraticEquationsBFCL/parallel_63: two solve_quadratic_equation calls — (5,6,1) and (3,2,1). Prompt names this function explicitly.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 788 OUT861

Tool Calling - Parallel Calls: Mathematics & Physics (set 3) 2 passed10 failed

16.7%

IN290 564 OUT12 029

×

AreaOfTwoCirclesInMetersBFCL/parallel_67: two geometry_calculate_area_circle calls — radii 5 and 10 in meters. unit required (tool default is 'cm'; the prompt states meters).1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 2

Failed

IN32 811 OUT1 178

×

AreaOfTwoTrianglesBFCL/parallel_69: two calculate_triangle_area calls — base/height 10x5 and 8x6.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2

Failed

IN27 349 OUT747

×

DerivativeOfTwoFunctionsAtValuesBFCL/parallel_73: two calculus_derivative calls — 3x^2+2x-1 at 5 (variable x, default) and 4y^3-3y^2+2y-1 at 3 (variable y).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN17 417 OUT740

×

DisplacementForThreeCarsBFCL/parallel_84: three calculate_displacement calls — (15,7,accel 3.5), (20,10,accel 2.0), (25,8,no acceleration). The third car travels "without any acceleration" so acceleration == 0 (also the default; omission accepted).1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 3

Failed

IN21 691 OUT744

✓

EuclideanNormForThreePointsBFCL/parallel_61: three math_hypot calls — (3,4) and (6,8) with z omitted/0, and (9,12,15).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 258 OUT264

×

FactorialOfThreeNumbersBFCL/parallel_60: three math_factorial calls — 5, 7, 9.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN12 647 OUT465

×

FinalSpeedForTwoFreeFallObjectsBFCL/parallel_85: two calculate_final_speed calls — initial_speed 0 for 10 s and initial_speed 5 for 7 s; gravity -9.81 (default, may be omitted).1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 7 expectedCount = 2

Failed

IN23 858 OUT787

✓

GcdViaNumberTheoryBFCL/parallel_78: two number_theory_gcd calls — (45,60) and (81,63). Prompt names this function explicitly.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 026 OUT159

×

IntegrateTwoFunctionsWithTwoMethodsBFCL/parallel_72: four integrate calls — x**3 on [2,5] and 2x**2+3x-1 on [-1,3], each with trapezoid (default, may be omitted) and simpson (explicit).1 AI query 16 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 4

Failed

IN25 925 OUT2 033

×

PrimeFactorsOfThreeNumbersBFCL/parallel_75: three number_analysis_prime_factors calls — 45, 100, 150.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN12 668 OUT393

×

SolveFourQuadraticsWithRootTypeBFCL/parallel_64: four solve_quadratic calls — (2,5,3) all roots, (1,-3,2) real, (4,-7,3) all roots, (1,2,1) real. 'all' is the default so it may be omitted; 'real' is explicitly requested and required.1 AI query 40 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 40 expectedCount = 4

Failed

IN77 689 OUT3 941

×

VelocityForTwoCyclingDaysBFCL/parallel_82: two calculate_velocity calls — 120 km in 5 h and 150 km in 6 h; unit km/h (default, may be omitted).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN22 225 OUT578

Tool Calling - Parallel Calls: Mixed Domains 1 passed9 failed

10.0%

IN279 811 OUT34 049

×

CalculateBmiForTwoPeopleBFCL/parallel_4: two calculate_bmi calls — 6 ft / 80 kg and 5.6 ft / 60 kg.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN11 276 OUT306

×

CalculateEmForceForTwoTimeIntervalsBFCL/parallel_1: two calculate_em_force calls — same field/area, d_time 4 then 10.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2

Failed

IN25 225 OUT1 170

×

CalculateFactorialForThreeNumbersBFCL/parallel_7: three math_factorial calls — 5, 10, 15.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN16 117 OUT495

×

CalculateResistanceForCopperAndAluminumBFCL/parallel_2: two calculate_resistance calls — copper then aluminum, same length/area. Tightened per methodology: the prompt explicitly names "resistivity of copper", so the first call must pass resistivity == "copper" (ground truth also accepts omission via the 'copper' default, but an explicitly-stated value must be supplied). Aluminum is non-default → required.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN11 324 OUT771

×

CalculateSalesTaxForThreeCitiesBFCL/parallel_6: three calculate_sales_tax calls — Chicago/IL $30.45, Sacramento/CA $52.33, Portland/OR $11.23.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN16 645 OUT1 010

×

FetchPopulationForCitiesStateAndCountryBFCL/parallel_8: four database_us_census_get_population calls — NYC (city), LA (city), Alaska (state), USA (country); year defaults to 2000 (may be omitted). Requires area-type inference.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4

Failed

IN17 395 OUT987

×

FindMovieShowingsForTwoMoviesBFCL/parallel_9: two find_movie_showing calls near San Diego — Tenet at 5 pm and No Time To Die at 7:30 pm.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN15 276 OUT432

×

GetSequenceAnd3DForThreeProteinsBFCL/parallel_3: three protein_info_get_sequence_and_3D calls — HbA1c, normal + rat hemoglobin. Tightened per methodology: the prompt explicitly asks for "their 3D models", so model_3d == true is required (ground truth also accepts omission via the default of true).1 AI query 299 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 299 expectedCount = 3

Failed

IN125 098 OUT27 749

×

ListShowRatingsOnNetflixAndHuluBFCL/parallel_5: two streaming_services calls — Netflix['Friends'] and Hulu['The Office','Stranger Things'] (Hulu show order unordered), both sorted by rating. Must group shows per service.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 2

Failed

IN34 189 OUT989

✓

PlayTwoArtistsForGivenDurationsBFCL/parallel_0: two spotify_play calls — Taylor Swift for 20 min and Maroon 5 for 15 min.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 266 OUT140

Tool Calling - Parallel Calls: Physical & Life Sciences (set 1) 3 passed9 failed

25.0%

IN214 425 OUT13 100

×

ElectricFieldForTwoChargesBFCL/parallel_94: two calculate_electric_field calls — charge 5/distance 2 and charge 3/distance 4. "Conducted in a vacuum" = the default permitivity → may be omitted or 0.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 971 OUT423

×

ElectromagneticForceInTwoMediaBFCL/parallel_96: two electromagnetic_force calls — same charges 5,10 and distance 2, differing by medium_permittivity. Vacuum call (8.854e-12) equals the default → may be omitted or 8.854e-12; the second medium (5e-12) is non-default → required.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN18 361 OUT1 659

×

EstimateTravelTimeForThreeRoutesBFCL/parallel_93: three route_estimate_time calls — (1) NYC->Miami via Philadelphia/Washington D.C./Atlanta, (2) NYC->Miami skipping Atlanta (via Philadelphia/Washington D.C.), (3) Philadelphia-> Miami via Washington D.C. Stop orderings from ground truth accepted. FLAG parallel_93: BORDERLINE. Ground truth allows many stop-list variants, some including the start city itself as a stop; a competent human's stop set is not fully deterministic. Kept BFCL-faithful by accepting the natural stop orderings (start city excluded from stops).1 AI query 6 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN13 478 OUT864

×

FinalSpeedForTwoDroppedObjectsBFCL/parallel_88: two calculate_final_speed calls — v0 0/height 10 and v0 5/height 20. gravity stated as 9.8 (the default) so it may be omitted or 9.8 per ground truth.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 2

Failed

IN13 506 OUT465

×

FinalVelocityForTwoCarsBFCL/parallel_87: two calculate_final_velocity calls — Car A (v0 0, a 6, t 10), Car B (v0 20, a 4, t 15).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 037 OUT477

×

FinalVelocityFromDistanceForTwoObjectsBFCL/parallel_86: two kinematics_final_velocity_from_distance calls — accel 5/dist 100 and accel 10/dist 200. "Both objects start from rest" → initial_velocity may be omitted or 0 (default).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 980 OUT572

×

FindVeganRestaurantsInThreeCitiesBFCL/parallel_91: three vegan_restaurant_find_nearby calls — LA, SF, Seattle, each with operating_hours 22 (prompt: "open until at least 22:00", a non-default value → required).1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3

Failed

IN24 321 OUT925

✓

GenerateFourTravelItinerariesBFCL/parallel_90: four travel_itinerary_generator calls — Tokyo/7/200/urban, Paris/10/150/history, Sydney/5/100/nature, Rome/12/180/culture. Tokyo's "urban" equals the default → omit or "urban".1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 791 OUT2 692

✓

GetDirectionsForRoundTripBFCL/parallel_89: three get_directions calls — SF->Palo Alto (fastest), Palo Alto->Golden Gate Bridge (scenic), Golden Gate Bridge->SF (fastest). route_type explicitly stated each time.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 516 OUT1 085

✓

MagneticFieldForTwoLoopsBFCL/parallel_95: two calculate_magnetic_field calls — current 10/radius 0.5 and current 15/radius 1.0. "Same as in free space" = default permeability → may be omitted or 0.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 518 OUT544

×

ResonantFrequencyForTwoCircuitsBFCL/parallel_97: two calculate_resonant_frequency calls — L 0.005/C 1e-07/round 3 and L 0.007/C 2e-07/round 4. Rounding is stated explicitly → required.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 2

Failed

IN35 247 OUT1 616

×

ShortestDrivingDistanceRoundTripBFCL/parallel_92: three get_shortest_driving_distance calls — NYC->LA, LA->Miami, Miami->NYC, all in miles (prompt explicitly "in miles" → non-default unit required).1 AI query 21 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 21 expectedCount = 3

Failed

IN44 699 OUT1 778

Tool Calling - Parallel Calls: Physical & Life Sciences (set 2) 12 failed1 broken

0.0%

IN317 452 OUT29 874

×

AbsolutePressureAtTwoAltitudesBFCL/parallel_102: two calc_absolute_pressure calls — gauge 2.5 with atm 1 (default; may be omitted) and gauge 2.5 with atm 0.85 (non-default → required).1 AI query 20 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 20 expectedCount = 2

Failed

IN61 101 OUT1 881

×

BoilingMeltingPointsAcrossSeaLevelsBFCL/parallel_100: four get_boiling_melting_points calls — water@0, iron@1000, water@1000, iron@0. FLAG parallel_100: BORDERLINE. The prompt ("water and iron at sea levels of 0 and 1000 meters respectively") reads most naturally as TWO calls (water@0, iron@1000), but the BFCL ground truth is the full 2x2 cross product (FOUR calls). Kept BFCL-faithful to the ground truth; a competent human could reasonably answer with two calls, so this item's expected call count is not fully deterministic.1 AI query 20 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 20 expectedCount = 2

Failed

IN37 286 OUT1 510

×

CellInfoDetailedAndBriefBFCL/parallel_108: two biology_get_cell_info calls — neuron with detailed=true (prompt: "detailed description"), muscle with detailed=false ("less detailed", the default).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN22 980 OUT1 214

×

DensityForTwoSubstancesBFCL/parallel_101: two calculate_density calls (mass/volume schema) — 10 kg / 2 m³ and 15 kg / 3 m³. unit "kg/m³" equals the default → may be omitted or "kg/m³".1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN18 021 OUT498

×

ElectricFieldStrengthAtSeveralDistancesBFCL/parallel_98: four calculate_electric_field_strength calls — charge 2, distances 0.5/1/2 in vacuum (default medium), plus distance 1 in air. The distance-1 vacuum and distance-1 air calls are distinguished by medium. Vacuum equals the default → may be omitted or "vacuum"; "air" is required.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4

Failed

IN19 961 OUT1 705

×

EntropyChangeForTwoConditionsBFCL/parallel_103: two entropy_change_calculate calls — substance A, mass 2, temps 25->75 and 10->50. pressure 1 equals the default → may be omitted or 1.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN24 000 OUT1 437

×

FetchDnaSequenceThreeWaysBFCL/parallel_106: three fetch_DNA_sequence calls for id XYZ123 — default fasta/0, then genbank format, then 500 bp upstream. Defaults (fasta, upstream 0) may be omitted.1 AI query 18 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 18 expectedCount = 3

Failed

IN37 473 OUT1 212

×

FunctionLookupForTwoMoleculesBFCL/parallel_110: two cell_biology_function_lookup calls — ATP in mitochondria and DNA in nucleus, both with specific_function=true (prompt asks for a specific function within the organelle).1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN23 320 OUT976

×

HeatCapacityForThreeConditionsBFCL/parallel_105: three calc_heat_capacity calls — 300K/2.5m³, 350K/2.5m³, 300K/1.5m³, all for air. "air" equals the default gas → may be omitted or "air".1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN13 700 OUT1 292

×

MolecularWeightForTwoCompoundsBFCL/parallel_111: two calculate_molecular_weight calls — Glucose (C6H12O6) and Sucrose (C12H22O11), both in grams/mole (prompt explicitly requests that unit; ground truth also accepts "g/mol").1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN23 270 OUT1 232

×

PhaseChangeEnergyForWaterBFCL/parallel_99: four thermo_calculate_energy calls — 500 g water, phase melting/freezing/ vaporization/condensation. Substance "water" equals the default → may be omitted or "water".1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN14 186 OUT1 401

!

ProteinSequencesForTwoGenesTwoSpeciesBFCL/parallel_107: four get_protein_sequence calls — BRCA1 and BRCA2 in Homo sapiens (default species, may be omitted) and in Pan troglodytes (non-default → required).1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Broken

IN8 712 OUT14 724

×

ProteinsInThreeCompartmentsBFCL/parallel_109: three cellbio_get_proteins calls — nucleus/mitochondria/cytoplasm, each with include_description=true (prompt asks for "a brief description of each protein").1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN13 442 OUT792

Tool Calling - Parallel Calls: Physical & Life Sciences (set 3) 12 failed1 broken

0.0%

IN261 759 OUT24 035

×

AnalyzeDnaForThreeMutationTypesBFCL/parallel_114: six analyze_dna_sequence calls — two sequences ("AGCTTAGCTA" and "AGCTTAGGCTA") against reference "AGCTTAGCTA", each checked for insertion (default, may be omitted), deletion, and substitution mutations.1 AI query 3 tool callscategory:ToolCallinglevel:VeryHardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 6

Failed

IN8 720 OUT738

×

DiabetesPredictionForFourPeopleBFCL/parallel_113: four diabetes_prediction calls for four people with distinct weight/height/activity.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4

Failed

IN20 275 OUT1 340

×

EntropyChangeIsothermalAndNotBFCL/parallel_104: two calculate_entropy_change calls — same 300->350 K and heat_capacity 4.18, once isothermal (the default true, may be omitted) and once not isothermal (false, required).1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2

Failed

IN29 620 OUT1 916

×

ForestGrowthForTwoRegionsBFCL/parallel_120: two forest_growth_forecast calls — Amazon Rainforest over 10 years and Boreal Forests of Canada over 20 years, excluding human impact (false = the default, may be omitted).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN18 147 OUT776

!

GenerateFourDnaSequencesBFCL/parallel_123: four generate_DNA_sequence calls — length 500 with a single-nucleotide preference of A, T, C, G respectively.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Broken

IN19 827 OUT10 495

×

GeneticSimilarityInTwoFormatsBFCL/parallel_115: four genetics_calculate_similarity calls — human vs chimpanzee and human vs gorilla, each in percentage (default, may be omitted) and fraction. species1 accepts "human"/"Human".1 AI query 20 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 20 expectedCount = 4

Failed

IN33 650 OUT1 759

×

GenotypeFrequenciesForThreeGenotypesBFCL/parallel_116: three calculate_genotype_frequency calls — allele frequency 0.7, genotypes AA/Aa/aa.1 AI query 15 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 15 expectedCount = 3

Failed

IN31 085 OUT1 180

×

IdentifyFourBirdsBFCL/parallel_119: four identify_bird calls — small blue/forest (default size, may be omitted), large black/lake, medium brown/desert, large green/tropical rainforest.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 4

Failed

IN20 135 OUT1 919

×

MutationTypeForTwoSnpsBFCL/parallel_112: two mutation_type_find calls — rs123456 in Homo sapiens (default species, may be omitted) and rs7891011 in Canis lupus familiaris (non-default → required).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 202 OUT438

×

PopulationDensityForChinaTwoYearsBFCL/parallel_117: two calculate_density calls (country/population schema) — China in 2000 (pop 1.267 billion) and 2010 (pop 1.341 billion), land area 9.597 million km² both times.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN21 182 OUT719

×

PrecipitationStatsForThreeTimeFramesBFCL/parallel_118: three ecology_data_precipitation_stats calls — Amazon rainforest for six_months, year, five_years.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 3

Failed

IN13 520 OUT1 021

×

TurtlePopulationForTwoYearsBFCL/parallel_121: two ecology_get_turtle_population calls — Galapagos Islands in 2015 and 2020, both with species info requested (species=true; years stated explicitly → required).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN13 199 OUT753

×

VehicleEmissionsForThreeVehiclesBFCL/parallel_122: three calculate_vehicle_emission calls — 15000 miles each: gas with default factor 1.4 (may be omitted), diesel with factor 2.7, EV with factor 0.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN19 197 OUT981

Tool Calling - Parallel Calls: Services & Lookups 1 passed11 failed

8.3%

IN205 096 OUT8 469

×

CalculateShortestAndScenicDistancesForTwoLegsBFCL/parallel_190: four calculate_shortest_distance calls — {NYC->LA, LA->Miami} x {Shortest, Scenic}.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN13 424 OUT763

×

FetchNewsForTwoTopicsInEuropeBFCL/parallel_192: two get_news calls — Climate Change and Artificial Intelligence, 5 articles each, region Europe. Region "Europe" explicitly requested -> required (accept "Europe"/"EU").1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 458 OUT761

×

FindLibrariesWithFacilitiesInThreeCitiesBFCL/parallel_191: three public_library_find_nearby calls — NY[Reading Room,Fiction], LA[Wi-Fi,Children Section], Chicago[Cafe,Reading Room]. Facilities compared as an unordered pair.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN18 179 OUT826

×

FindParksWithAmenitiesInThreeCitiesBFCL/parallel_189: three parks_find_nearby calls — NY[Tennis Court,Picnic Area], LA[Playground,Running Track], Chicago[Tennis Court,Playground]. Amenities compared as an unordered pair.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN18 201 OUT990

×

FindUpcomingEventsInThreeCitiesBFCL/parallel_194: three event_finder_find_upcoming calls — jazz/LA(14d), rock/Chicago(10d), classical/Boston(7d). All day windows explicitly stated -> days_ahead required (7 for Boston, which also equals the default).1 AI query 21 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 21 expectedCount = 3

Failed

IN43 983 OUT1 779

×

GetCurrentWeatherForFourCitiesBFCL/parallel_199: four get_current_weather calls — New York, Los Angeles, London, Tokyo, with both temperature and humidity included. Prompt explicitly asks to include both -> flags == true (also the default).1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 4

Failed

IN13 349 OUT621

×

GetDetailedBriefsForTwoMoviesBFCL/parallel_195: two movie_details_brief calls — Inception and The Dark Knight, extra_info true. Prompt asks for Director/Cast/Awards -> extra_info == true required.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN22 020 OUT711

×

GetLawsuitDetailsForTwoCasesBFCL/parallel_196: two get_lawsuit_details calls — case 12345 (NY Supreme Court, with verdict) and case 67890 (LA Superior Court, no verdict). "would like the verdict" -> true; "do not need" -> null/false.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 680 OUT462

×

GetLawsuitInfoForTwoCasesByYearBFCL/parallel_197: two lawsuit_info calls — 12345ABC (2018, New York) and 67890XYZ (2019, California). Years are explicitly stated -> required. First location "New York" may be omitted (matches the default).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN17 443 OUT579

×

GetStockPricesForTwoCompaniesOverDaysBFCL/parallel_179: two get_stock_price (by-days variant) calls — Apple on NASDAQ for 10 days, Microsoft on NYSE for 15 days. Both exchanges are explicitly named -> required (NASDAQ / NYSE). The tool's 'company' field is a ticker symbol; company names are accepted as synonyms.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 464 OUT329

×

SearchLawsuitsForTwoEntitiesInCaliforniaBFCL/parallel_198: two lawsuit_search calls — Google/Santa Clara and Facebook/San Mateo, state California. Prompt states "both in the state of California" -> state required (accept "California"/"CA").1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 4 expectedCount = 2

Failed

IN12 518 OUT344

✓

SendTwoEmailsWithCcAndBccBFCL/parallel_193: two send_email calls — Project Update to john.doe (cc manager, bcc hr) and Meeting Reminder to jane.doe (cc assistant, bcc hr). Bodies matched exactly.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 377 OUT304

Tool Calling - Parallel Calls: Statistics & Finance 1 passed12 failed

7.7%

IN368 633 OUT19 234

×

CalculateBinomialProbabilitiesForBasketballShotsBFCL/parallel_163: three calc_binomial_prob calls — (10,6,0.6), (10,6,0.5), (15,6,0.5).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN20 047 OUT1 500

×

CalculateCompoundInterestForThreePeriodsBFCL/parallel_170: three calculate_compound_interest calls — principal 5000, rate 0.025, time 2/3/5. Prompt states "compounded quarterly, 4 times in a year" -> n == 4 required (tightened; also ground truth).1 AI query 27 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 27 expectedCount = 3

Failed

IN64 541 OUT2 285

×

CalculateDiscountedCashFlowForTwoBondsBFCL/parallel_169: two calculate_discounted_cash_flow calls — (50,5,0.05) and (60,7,0.04). Prompt explicitly states "face value of the bond is $1000" for both -> face_value == 1000 required (ground truth also accepts omission via the default, but an explicitly-stated value must be supplied).1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 16 expectedCount = 2

Failed

IN50 722 OUT2 569

×

CalculateEventProbabilitiesForCardDeckBFCL/parallel_162: three probability_of_event calls — Ace 4/52 as decimal, heart 13/52 as decimal, red card 26/52 as ratio. format_as_ratio: "as a decimal" -> false or omitted; "as a ratio" -> true (required).1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 18 expectedCount = 3

Failed

IN39 867 OUT1 437

×

CalculateQuarterlyDividendForThreeScenariosBFCL/parallel_168: three finance_calculate_quarterly_dividend_per_share calls — (5M,2M), (6M,2M), (6M,2.5M).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 9 expectedCount = 3

Failed

IN20 201 OUT952

×

CalculateReturnOnEquityForTwoCompaniesBFCL/parallel_171: two calculate_return_on_equity calls — (1M,5M,200k) and (2M,10M, no dividends). Second company "did not pay any dividends" -> dividends_paid null or 0.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN24 700 OUT1 020

×

CalculateRoiForThreeStocksBFCL/parallel_174: three calculate_return_on_investment calls — (150,180,20), (200,210,30), (250,300,40). Dividends are explicitly stated per stock -> required.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 12 expectedCount = 3

Failed

IN26 116 OUT1 188

✓

ComputeRSquaredForTwoRegressionModelsBFCL/parallel_167: two linear_regression_get_r_squared calls on the same CSV — swap education/credit_score as dependent variable. Independent-variable order follows the prompt/ground truth.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 934 OUT557

×

PredictFutureValueForTwoInvestmentsBFCL/parallel_172: two finance_predict_future_value calls — (5000,0.05,10) and (7000,0.04,15). Prompt states "compound interest annually" -> compounding_periods_per_year == 1 required (tightened; also the default).1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 2

Failed

IN30 664 OUT1 370

×

PredictHousePricesForThreeHousesBFCL/parallel_166: three predict_house_price calls — NY 2500/3/2000, LA 3000/3/2005, Chicago 2000/2/1995. The second house is described as "similar" with only area/year/location changed, so rooms == 3 per ground truth.1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 10 expectedCount = 3

Failed

IN25 386 OUT1 144

×

PredictProfitForTwoFundsBFCL/parallel_173: two investment_predictProfit calls — (5000,0.07,5) and (8000,0.05,7).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 6 expectedCount = 2

Failed

IN18 953 OUT1 165

×

RunChiSquaredTestForTwoContingencyTablesBFCL/parallel_164: two chi_squared_test calls — tables [45,55,35,65] and [30,70,50,50], alpha 0.05. Prompt states "significance level of 0.05 for both" -> alpha == 0.05 required (tightened). Table order is fixed.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 8 expectedCount = 2

Failed

IN24 405 OUT2 002

×

RunTTestAtTwoSignificanceLevelsBFCL/parallel_165: two t_test calls — same datasets, alpha 0.05 then 0.01 (both stated -> required).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == expectedCount Values: response.ToolCalls.Count = 3 expectedCount = 2

Failed

IN14 097 OUT2 045

Tool Calling - Simple: Arts, Venues & Music (set 1) 8 failed

0.0%

IN1 193 334 OUT15 312

×

SimpleConcertDetailsBeyonceBFCL/simple_python_286 (concert.get_details): artist=Beyonce, location=San Diego, date=April 2022.1 AI query 17 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 17

Failed

IN80 018 OUT1 621

×

SimpleConcertInfoPopNyBFCL/simple_python_284 (concert_info.get): location=New York, date=next month, genre=Pop.1 AI query 47 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 47

Failed

IN324 230 OUT3 981

×

SimpleGenerateMelodyC4BFCL/simple_python_291 (music_generator.generate_melody): key=C, start_note=C4, length=16, tempo=120 or omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 040 OUT267

×

SimpleKeySignatureCsharpBFCL/simple_python_298 (music_theory.key_signature): key='C#', scale_type=major or omitted.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN347 516 OUT3 917

×

SimpleMuseumHoursLouvreBFCL/simple_python_273 (museum.get_hours): name='Louvre Museum', location=Paris, day omitted or Monday.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN20 543 OUT339

×

SimplePaintNeededWall30x12BFCL/simple_python_259 (calculate_paint_needed): coverage_rate=400, length=30, height=12.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN16 981 OUT368

×

SimpleReligionHistoryChristianity16cBFCL/simple_python_252 (get_religion_history): religion=Christianity, century=16, sort_by=importance, count=10.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN373 210 OUT4 330

×

SimpleSculptureInfoPlensaBFCL/simple_python_263 (get_sculpture_info): artist_name='James Plensa', detail=true.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN16 796 OUT489

Tool Calling - Simple: Arts, Venues & Music (set 2) 8 failed

0.0%

IN465 485 OUT8 857

×

SimpleComposeMelodyCFGBFCL/simple_python_292 (compose_melody): progression=[C,F,G], measures=4, instrument=Piano or omitted.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN42 074 OUT883

×

SimpleConcertFindDetailsWeekndBFCL/simple_python_290 (concert.find_details): artist=The Weeknd, month=December, year=2022 or omitted.1 AI query 35 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 35

Failed

IN206 966 OUT2 820

×

SimpleFindConcertRockChicagoBFCL/simple_python_285 (find_concert): location=Chicago, price=100, genre=Rock.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN41 551 OUT678

×

SimpleMuseumInfoMetBFCL/simple_python_274 (museum_info): museum_name=Metropolitan Museum of Art, info_type=opening_hours or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN20 501 OUT401

×

SimpleMusicalScaleCsharpBFCL/simple_python_299 (musical_scale): key='C#'/'C sharp', scale_type=major or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 597 OUT373

×

SimplePaintRequirementWallExclusionBFCL/simple_python_260 (paint_requirement.calculate): area {width=20,height=12}, paint_coverage=350, exclusion {type=window, area=15}.1 AI query 18 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 18

Failed

IN91 397 OUT2 271

×

SimpleReligionHistoryChristianity300To400BFCL/simple_python_254 (get_religion_history): religion=Christianity, start_year=300, end_year=400, event_type=all or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN20 810 OUT785

×

SimpleSculptureDetailsDavidBFCL/simple_python_264 (sculpture.get_details): artist=Michelangelo, title=David, detail=size.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN20 589 OUT646

Tool Calling - Simple: Arts, Venues & Music (set 3) 1 passed8 failed

11.1%

IN885 787 OUT12 316

✓

SimpleChordProgressionKeyBFCL/simple_python_297 (music.theory.chordProgression): progression=[I,V,vi,IV], bools any.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 697 OUT354

×

SimpleCompoundInterest10kBFCL/simple_python_269 (calculate_compound_interest): principle=10000, interest_rate=0.05, time=10, compounds_per_year=1 or omitted.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN35 468 OUT1 021

×

SimpleConcertSearchClassicalLaBFCL/simple_python_287 (concert.search): genre=classical, location=Los Angeles/LA, date='this weekend', price_range=cheap.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN377 045 OUT4 579

×

SimpleCreateMixCMajorBFCL/simple_python_293 (music_composer.create_mix): scale='C Major', note_duration=quarter, track_length=180.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 262 OUT229

×

SimpleGenerateCircleImageRedBFCL/simple_python_256 (generate_circle_image): radius=50, color=Red, background=white or omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 196 OUT127

×

SimpleMuseumWorkingHoursLouvreBFCL/simple_python_276 (museum_working_hours.get): museum=Louvre, location=Paris, day=Monday or omitted.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN50 877 OUT914

×

SimpleRetrieveReligionInfoBuddhismBFCL/simple_python_253 (retrieve_religion_info): religion_name=Buddhism, detail_level=full.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN25 764 OUT1 341

×

SimpleSculptureSearchChicago19cBFCL/simple_python_265 (sculpture_search): location=Chicago, time_frame='19th century', material=all or omitted.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN359 297 OUT3 501

×

SimpleTimeDifferenceSfSydneyBFCL/simple_python_250 (get_time_difference): place1=San Francisco/SF, place2=Sydney.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 181 OUT250

Tool Calling - Simple: Arts, Venues & Music (set 4) 9 failed

0.0%

IN1 032 125 OUT15 209

×

SimpleBuildingDimensionsEmpireStateBFCL/simple_python_270 (building.get_dimensions): building_name=Empire State Building, unit=feet.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 656 OUT192

×

SimpleConcertFindNearbySeattleBFCL/simple_python_289 (concert.find_nearby): location=Seattle, genre=jazz.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN16 746 OUT457

×

SimpleCreateChordProgressionCBFCL/simple_python_294 (music_generation.create_chord_progression): key=C, chords=4, progression_type=major or omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 188 OUT283

×

SimpleEarliestReferenceJesusBFCL/simple_python_251 (get_earliest_reference): name='Jesus Christ', source='historical records'.1 AI query 19 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 19

Failed

IN89 646 OUT1 400

×

SimpleFindInstrumentAcousticBFCL/simple_python_280 (find_instrument): budget=1000, type=acoustic, make omitted.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN28 560 OUT639

×

SimpleGuitarPriceLesPaulBFCL/simple_python_283 (guitar_price.find): model='Gibson Les Paul', condition=Excellent, location=Chicago.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN20 702 OUT715

×

SimpleIdentifyColorSeaGreenBFCL/simple_python_257 (identify_color_rgb): color_name='Sea Green', standard=basic or omitted.1 AI query 24 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 24

Failed

IN120 731 OUT1 950

×

SimpleMuseumInfoBritishDateBFCL/simple_python_277 (museum_info): museum='The British Museum', date=2023-06-20, information=[opening_hours,ticket_price] (either order).1 AI query 50 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN378 954 OUT5 372

×

SimpleSculptureValueThinkerBFCL/simple_python_266 (get_sculpture_value): sculpture='The Thinker', artist=Rodin.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN353 942 OUT4 201

Tool Calling - Simple: Arts, Venues & Music (set 5) 1 passed8 failed

11.1%

IN759 860 OUT10 981

×

SimpleCircleDimensionsRadius5BFCL/simple_python_272 (calculate_circle_dimensions): radius=5.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 212 OUT162

✓

SimpleDrawRectangleRedBFCL/simple_python_261 (draw_rectangle): width=20, height=10, color=red.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 416 OUT89

×

SimpleFindFluteYamahaBFCL/simple_python_282 (find_flute): brand=Yamaha, specs=[open hole, C foot, silver headjoint].1 AI query 40 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 40

Failed

IN263 671 OUT3 584

×

SimpleGenerateScaleProgressionCBFCL/simple_python_296 (music_generator.generate_scale_progression): key=C, tempo=80, duration=4, scale_type=major or omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 830 OUT246

×

SimpleInstrumentDetailsPianoYamahaBFCL/simple_python_278 (get_instrument_details): instrument=piano, manufacturer=Yamaha, features=[price,rating] (either order).1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN42 973 OUT969

×

SimpleMetTopArtworksBFCL/simple_python_275 (metropolitan_museum.get_top_artworks): number=5, sort_by=popularity or omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 328 OUT949

×

SimpleMixPaintYellowBlueBFCL/simple_python_258 (mix_paint_color): color1=yellow, color2=blue, lightness=60.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 794 OUT154

×

SimplePapalBiographyInnocentIIIBFCL/simple_python_255 (religious_history.get_papal_biography): papal_name=Innocent III, include_contributions=true.1 AI query 38 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 38

Failed

IN236 780 OUT2 482

×

SimpleSculptureLocatorMichelangeloBFCL/simple_python_268 (sculpture_locator.find_by_artist): artist=Michelangelo, material=Marble, location=Rome.1 AI query 29 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 29

Failed

IN163 856 OUT2 346

Tool Calling - Simple: Arts, Venues & Music (set 6) 1 passed6 failed

14.3%

IN99 551 OUT2 198

×

SimpleAnalyzeStructureB1004BFCL/simple_python_271 (analyze_structure): building_id=B1004, floors=[2,3,4], mode=dynamic.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 206 OUT240

×

SimpleConcertBookingEminemBFCL/simple_python_288 (concert_booking.book_ticket): artist=Eminem, city=New York City, num_tickets=2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 137 OUT164

×

SimpleFindExhibitionNyBFCL/simple_python_267 (find_exhibition): location='New York City, NY', art_form=sculpture, month omitted, user_ratings=high.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN23 522 OUT520

×

SimpleInstrumentInfoViolin1721BFCL/simple_python_281 (get_instrument_info): name=Violin, maker=Stradivarius, year=1721.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN19 724 OUT546

×

SimpleInstrumentPriceStratocasterBFCL/simple_python_279 (instrument_price.get): brand=Fender, model='American Professional II Stratocaster', finish=Rosewood.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN16 011 OUT305

✓

SimpleModifyPaintingOilBFCL/simple_python_262 (modify_painting): size='12x18', medium=oil, dominant_color=red.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 987 OUT98

×

SimpleSongLyricsBohemianBFCL/simple_python_295 (get_song_lyrics): song_title='Bohemian Rhapsody', artist_name=Queen, lang=English or omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN15 964 OUT325

Tool Calling - Simple: Food, Travel & Shopping (set 1) 1 passed7 failed

12.5%

IN656 475 OUT10 436

✓

SimpleCookingConvertPoundsToOuncesBFCL/simple_python_365 (cooking_conversion.convert): 2 pounds->ounces of butter.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 470 OUT120

×

SimpleCurrencyExchangeUsdToCadBFCL/simple_python_388 (currency_exchange.convert): USD->CAD, amount=500.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 186 OUT381

×

SimpleGroceryStoreOrganicHoustonBFCL/simple_python_369 (grocery_store.find_nearby): Houston, categories {Organic,Fruits,Vegetables} any order.1 AI query 24 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 24

Failed

IN125 667 OUT1 944

×

SimpleHotelBookPetFriendlyManhattanBFCL/simple_python_380 (hotel_booking): Manhattan, single, 3 nights, 2023-03-10, pet_friendly.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN25 848 OUT643

×

SimpleMapsDistanceEiffelLouvreBFCL/simple_python_394 (maps.get_distance_duration): Eiffel Tower -> Louvre Museum, traffic omitted/false.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 166 OUT292

×

SimpleRecipeFindByDietMealBFCL/simple_python_353 (find_recipes): diet=gluten-free, meal_type=dinner, ingredients omitted.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN352 055 OUT3 776

×

SimpleRestaurantFinderItalianNYCBFCL/simple_python_361 (restaurant_finder): city=NYC, cuisine=Italian, diet=Gluten-free.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 448 OUT352

×

SimpleTimeZoneLondon24hBFCL/simple_python_376 (time_zone_converter): city=London, country=UK, display_format=24h.1 AI query 20 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 20

Failed

IN98 635 OUT2 928

Tool Calling - Simple: Food, Travel & Shopping (set 2) 9 failed

0.0%

IN1 197 841 OUT14 607

×

SimpleBestSushiPlacesTokyoBFCL/simple_python_362 (get_best_sushi_places): city=Tokyo, top=5, review_rate=4.0.1 AI query 33 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 33

Failed

IN205 770 OUT2 557

×

SimpleCurrencyConverterUsdToGbpBFCL/simple_python_389 (currency_converter): USD->GBP, amount=200.0.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 636 OUT325

×

SimpleCurrentTimeSydneyCityBFCL/simple_python_377 (get_current_time): city=Sydney, country=Australia, format omitted/HH:MM:SS.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN31 520 OUT688

×

SimpleDistanceCalcNyToBostonTerrainBFCL/simple_python_397 (distance_calculator.calculate): New York -> Boston, consider_terrain=true.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN22 784 OUT363

×

SimpleHiltonAvailabilityParisBFCL/simple_python_381 (hilton_hotel.check_availability): Paris, 2023-04-04..08, 2 adults, chain Hilton/omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 494 OUT564

×

SimpleRecipeCaloriesBeefLasagnaBFCL/simple_python_355 (recipe_info.get_calories): website=Foodnetwork.com, recipe=Beef Lasagna, meal_time omitted.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN370 948 OUT4 344

×

SimpleRecipeUnitTbspToTspBFCL/simple_python_366 (recipe.unit_conversion): 2 tablespoon->teaspoon, precision omitted/1.1 AI query 25 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 25

Failed

IN140 697 OUT1 852

×

SimpleSafewayOrderPaloAltoBFCL/simple_python_370 (safeway.order): Palo Alto, [olive oil, rice], [3,1]. EXPECTED MODEL MISS (kept, do not loosen): "a five pound bag of rice" = quantity 1 (5 is the weight, not the count); models tend to send quantity 5. This is a genuine model error the test correctly catches.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 010 OUT345

×

SimpleVeganSoupRecipeUnder30BFCL/simple_python_354 (get_vegan_recipe): dish_type=soup, cooking_time=30, ingredient_preference omitted.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN364 982 OUT3 569

Tool Calling - Simple: Food, Travel & Shopping (set 3) 1 passed7 failed

12.5%

IN546 934 OUT8 530

×

SimpleBookHotelHiltonChicagoBFCL/simple_python_382 (book_hotel): Hilton Hotel, Chicago, single, 2022-12-10, 2 nights.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 040 OUT260

×

SimpleCookingTimeRoastChickenBFCL/simple_python_368 (calculate_cooking_time): weight_kg=1.5, method omitted/roast, temp omitted/180.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN47 786 OUT838

×

SimpleCurrencyConvertEurToCadBFCL/simple_python_390 (currency_conversion.convert): 150 EUR->CAD.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN363 069 OUT4 567

×

SimpleCurrentTimeSydneyLocationBFCL/simple_python_379 (get_current_time): location=Sydney, country=Australia, timezone omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 126 OUT319

×

SimpleParkingLotCentralParkBFCL/simple_python_395 (parking_lot.find_nearest): Central Park NY, radius=2, type omitted/public.1 AI query 13 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 13

Failed

IN57 627 OUT911

×

SimpleRecipeFinderVeganServes2BFCL/simple_python_356 (recipe_finder.find): servings=2, diet=vegan, prep_time=30.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 206 OUT1 118

×

SimpleRestaurantClosestSushiBostonBFCL/simple_python_363 (restaurant_search.find_closest): Boston, Sushi, amenities=[Patio].1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 587 OUT422

✓

SimpleWholeFoodsPriceLABFCL/simple_python_371 (whole_foods.check_price): Los Angeles, [tomatoes, lettuce].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 493 OUT95

Tool Calling - Simple: Food, Travel & Shopping (set 4) 7 failed

0.0%

IN275 914 OUT13 666

×

SimpleBookRoomThePlazaBFCL/simple_python_383 (book_room): The Plaza, single, 2 nights.1 AI query 11 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN126 321 OUT9 415

×

SimpleConvertCurrencyJpyToUsdBFCL/simple_python_393 (convert_currency): 20000 JPY->USD.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN35 994 OUT824

×

SimpleFindRestaurantItalianBrooklynBFCL/simple_python_364 (find_restaurant): Brooklyn, Italian, Gluten-free.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN12 518 OUT239

×

SimpleGetRecipeVeganChocCakeBFCL/simple_python_357 (get_recipe): dish_name=(vegan) chocolate cake, diet_preference=vegan.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN49 178 OUT1 292

×

SimpleHospitalLocateDenverBFCL/simple_python_396 (hospital.locate): Denver CO, radius=5, department=Pediatrics.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN12 528 OUT430

×

SimpleTimezoneConvertNyToLondonBFCL/simple_python_378 (timezone.convert): 3pm, New York -> London.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN19 693 OUT916

×

SimpleWholeFoodsTopBrandsBananasBFCL/simple_python_372 (whole_foods.find_top_brands): product=bananas, number omitted/5, organic=true.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN19 682 OUT550

Tool Calling - Simple: Food, Travel & Shopping (set 5) 6 failed

0.0%

IN129 139 OUT3 242

×

SimpleExchangeRateFeeGbpJpyBFCL/simple_python_391 (get_exchange_rate_with_fee): GBP->JPY, fee=0.02.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN23 188 OUT477

×

SimpleHotelBookingBookParisBFCL/simple_python_384 (hotel_booking.book): Paris, 07-10-2022..07-20-2022, 2 adults, 1 child, Standard/omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 176 OUT320

×

SimpleMuseumHoursMetSaturdayBFCL/simple_python_398 (get_museum_hours): museum=Metropolitan Museum of Art/The Met, day=Saturday.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN15 666 OUT322

×

SimpleRecipeSearchGfCookieBFCL/simple_python_358 (recipe_search): diet=[gluten free], time_limit=30, dish=cookie.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN49 047 OUT1 258

×

SimpleRestaurantSearchRatedNYCBFCL/simple_python_399 (restaurant_search): NYC, Italian, rating=4, accepts_credit_cards=true.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN15 866 OUT488

×

SimpleWalmartPurchaseSanJoseBFCL/simple_python_373 (walmart.purchase): San Jose, [apples, rice, (bottled) water], pack_size=[1,1,12].1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN16 196 OUT377

Tool Calling - Simple: Food, Travel & Shopping (set 6) 5 failed

0.0%

IN492 253 OUT7 379

×

SimpleGameHighestScoringPlayerBFCL/simple_python_350 (get_highest_scoring_player): game=Valorant, season=2022.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN14 876 OUT264

×

SimpleHotelBookRoomKingLABFCL/simple_python_385 (hotel_bookings.book_room): LA, king size, 15-10-2023, 2 nights, rooms omitted/1.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN8 624 OUT259

×

SimpleLatestExchangeGbpJpyBFCL/simple_python_392 (latest_exchange_rate): GBP->JPY, amount omitted/1.0.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN327 785 OUT4 128

×

SimpleNutritionalInfoAvocadoBFCL/simple_python_374 (grocery_info.nutritional_info): Walmart, avocado, [Protein,Calories,Carbohydrates].1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN11 702 OUT392

×

SimpleRecipeSearchVegPastaBFCL/simple_python_359 (recipe_search): dietary_restriction=Vegetarian, ingredients=[pasta,cheese], servings=2.1 AI query 26 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 26

Failed

IN129 266 OUT2 336

Tool Calling - Simple: Food, Travel & Shopping (set 7) 1 passed3 failed

25.0%

IN96 383 OUT2 258

✓

SimpleBookHotelLuxuryParadiseBFCL/simple_python_386 (book_hotel): Hotel Paradise, Las Vegas, luxury, 05-12-2022, 3 days, view=city.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 268 OUT148

×

SimpleFindRecipeCaloriesCarbonaraBFCL/simple_python_360 (find_recipe): recipeName=pasta carbonara, maxCalories=500.1 AI query 16 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN62 868 OUT1 465

×

SimpleGameMultiplayerFinderBFCL/simple_python_351 (multiplayer_game_finder): platform=Windows 10, rating=4.5, genre omitted/Action.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN17 330 OUT409

×

SimpleWalmartPriceCheckPumpkinsBFCL/simple_python_375 (walmart.check_price): [pumpkins,eggs], quantities [3,24] or [3,2], store omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 917 OUT236

Tool Calling - Simple: Food, Travel & Shopping (set 8) 1 passed2 failed

33.3%

IN108 550 OUT2 930

×

SimpleFindRecipeVeganBrowniesBFCL/simple_python_367 (find_recipe): dietary_restrictions=vegan, recipe_type=dessert (brownies), time=30.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 228 OUT615

×

SimpleGameAverageUserScoreBFCL/simple_python_352 (gamespot.getAverageUserScore): game=Zelda BotW, platform=Nintendo Switch/all platforms.1 AI query 22 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 22

Failed

IN93 426 OUT2 110

✓

SimpleHotelBookingPlazaRangeBFCL/simple_python_387 (hotel_booking): Plaza Hotel, New York NY, 2022-06-01..2022-06-04, rooms omitted/1.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 896 OUT205

Tool Calling - Simple: Knowledge, Places & History (set 1) 10 failed

0.0%

IN317 536 OUT6 627

×

SimpleAirQualityChicagoBFCL/simple_python_203 (get_air_quality): location=Chicago, detail=true.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN22 289 OUT438

×

SimpleEvolutionScientistBFCL/simple_python_242 (get_scientist_for_discovery): discovery=Theory of Evolution.1 AI query 12 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN54 787 OUT894

×

SimpleGermanWarEventsBFCL/simple_python_231 (history.get_key_events): country=Germany/DE, 1871-1945, event_type=[War].1 AI query 14 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 14

Failed

IN68 713 OUT1 905

×

SimpleKingOfFranceBFCL/simple_python_230 (historic_leader_search): location=France, date=1510, title omitted or King.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN22 460 OUT410

×

SimplePandaPopulationBFCL/simple_python_201 (estimate_population): species=panda(s), country=China/CN, year omitted or 2024.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 622 OUT226

×

SimplePersonalityTraitsTypeBFCL/simple_python_227 (get_personality_traits by type): type=ENFJ, traits=[strengths, weaknesses].1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN31 774 OUT869

×

SimplePresidentCivilWarBFCL/simple_python_238 (us_history.get_president): event=American Civil War, year in 1861-1865.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN18 290 OUT245

×

SimpleRestaurantNearbyBFCL/simple_python_204 (restaurant.find_nearby): location=Seattle[, WA], cuisine=Chinese, max_distance=10.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN50 534 OUT961

×

SimpleTordesillasDateBFCL/simple_python_233 (european_history.get_event_date): event=Treaty of Tordesillas, format=YYYY.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 592 OUT142

×

SimpleTrafficBostonNewYorkBFCL/simple_python_205 (get_traffic_info): start=Boston, end=New York/NYC, mode omitted or driving.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN22 475 OUT537

Tool Calling - Simple: Knowledge, Places & History (set 2) 10 failed

0.0%

IN1 344 725 OUT16 203

×

SimpleBitcoinNewsBFCL/simple_python_210 (get_news): topic=Bitcoin, quantity=5, region omitted or US.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 288 OUT401

×

SimpleEuropeanWars19cBFCL/simple_python_234 (history_eu.fetch_events): century=19, region=any of N/S/E/W, category=Wars. EXPECTED MODEL MISS (kept, do not loosen): region is an enum (Northern/Southern/Eastern/Western); the prompt says only "European", so any of the four is accepted, but models often send the invalid value "Europe". Failing to respect the enum is a genuine model error the test correctly catches.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN373 730 OUT4 518

×

SimpleLibraryNearbyBFCL/simple_python_209 (public_library.find_nearby): location=Boston MA/Massachusetts, facilities=[Fiction, Wi-Fi] (any order).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 542 OUT279

×

SimpleMonarchOfEnglandBFCL/simple_python_232 (monarch.getMonarchOfYear): location=England/ENG, year=1800, fullName=true.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 345 OUT300

×

SimpleNeutronDiscovererBFCL/simple_python_243 (get_discoverer): discovery=neutron, detail=true.1 AI query 25 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 25

Failed

IN134 482 OUT1 766

×

SimpleParksNearbyBFCL/simple_python_206 (parks.find_nearby): location=London[, UK], amenities=[Tennis Court].1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN31 633 OUT631

×

SimplePersonalityTraitsHobbyBFCL/simple_python_228 (get_personality_traits by hobby): hobby=jogging, trait_count=3.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN363 326 OUT4 126

×

SimplePopulationGrowthBFCL/simple_python_221 (population_growth_estimate): location=London, years=5, rate omitted or 1.2.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN22 248 OUT356

×

SimplePresidentDuringEventBFCL/simple_python_241 (US_President_During_Event): event=Civil War, country omitted or USA.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN354 011 OUT3 579

×

SimpleTreatyLisbonDateBFCL/simple_python_235 (get_event_date): event=Treaty of Lisbon (variants), location omitted or Lisbon[, Portugal].1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 120 OUT247

Tool Calling - Simple: Knowledge, Places & History (set 3) 2 passed8 failed

20.0%

IN973 433 OUT11 646

×

SimpleAppleStockInfoBFCL/simple_python_212 (get_stock_info): company=Apple Inc., detail_level=detailed, market omitted or NASDAQ.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 016 OUT341

×

SimpleCivilWarStartDateBFCL/simple_python_236 (us_history.get_event_info): event=American Civil War/Civil War, specific_info=Start Date.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 288 OUT321

✓

SimpleFlightBookBFCL/simple_python_213 (flight.book): departure=San Francisco/SF, destination=London, date=2022-04-27 (variants), time=afternoon, direct=true.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN7 278 OUT185

×

SimpleGravitationPublicationYearBFCL/simple_python_244 (publication_year.find): author=Isaac Newton, work=Law of Universal Gravitation (variants), location omitted or all.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 085 OUT302

×

SimpleMovieBriefBFCL/simple_python_215 (movie_details.brief): title=Interstellar, extra_info omitted or false.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN362 981 OUT3 689

×

SimplePresidentIn1861BFCL/simple_python_239 (US_president.in_year): year=1861, full_name omitted or true.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN375 170 OUT4 252

×

SimpleRadiumDiscovererBFCL/simple_python_245 (discoverer.get): element_name=radium (quote variants), year omitted or 0, first omitted or true.1 AI query 18 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 18

Failed

IN92 081 OUT1 242

✓

SimpleSendEmailBFCL/simple_python_211 (send_email): to=john.doe@example.com, subject=Meeting, body (opt trailing period), cc/bcc empty.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 280 OUT108

×

SimpleShortestDistanceBFCL/simple_python_207 (calculate_shortest_distance): start=New York (variants), end=Miami (variants), route_preference=Shortest.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 038 OUT244

×

SimpleUpcomingConcertsBFCL/simple_python_214 (event_finder.find_upcoming): location=New York (variants), genre=Rock, days_ahead=30.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN52 216 OUT962

Tool Calling - Simple: Knowledge, Places & History (set 4) 1 passed9 failed

10.0%

IN950 247 OUT12 400

×

SimpleBmiCalcBFCL/simple_python_222 (calculate_bmi): weight=70, height=180, unit omitted or metric.1 AI query 16 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN79 239 OUT1 174

×

SimpleCarEmissionsBFCL/simple_python_200 (calculate_emissions): distance=12000, fuel_type=gas, fuel_efficiency omitted or 25.0, efficiency_reduction omitted or 0.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN388 869 OUT4 928

×

SimpleFmriAnalyzeBFCL/simple_python_217 (fMRI.analyze): data_source path, sequence_type=multi-band, smooth=6, voxel_size=2.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 024 OUT311

×

SimpleGetDirectionsBFCL/simple_python_208 (map_service.get_directions): start=New York/NYC, end=Los Angeles/LA, avoid=[highways, tolls] (any order).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 000 OUT560

×

SimpleGravityDiscoveryBFCL/simple_python_246 (science_history.get_discovery_details): discovery=Gravity, method_used omitted or default.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN360 596 OUT3 524

×

SimpleGroupDynamicsBFCL/simple_python_223 (group_dynamics.pattern): total=50, extroverts=15, introverts=35.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 309 OUT596

×

SimpleMostFollowedBFCL/simple_python_224 (social_media_analytics.most_followed): topic=psychology, sub_topics=[behaviour, group dynamics], region omitted or all.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 049 OUT460

×

SimpleNeuronCoordinatesBFCL/simple_python_219 (get_neuron_coordinates): neuron_type=GABA, brain_region=All/all part of the brain/entire brain.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 126 OUT466

✓

SimplePresidentIn1940BFCL/simple_python_240 (history_api.get_president_by_year): year=1940, full_term_only omitted/true/false.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 155 OUT129

×

SimpleSentimentAnalysisBFCL/simple_python_216 (sentiment_analysis): text=the review, language=english/en.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 880 OUT252

Tool Calling - Simple: Knowledge, Places & History (set 5) 10 failed

0.0%

IN1 504 688 OUT21 057

×

SimpleBigFiveScoresBFCL/simple_python_229 (get_bigfive_scores): characteristics=[efficient, organized, easy going, compassionate], scale omitted or medium.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN28 605 OUT596

×

SimpleChristianityHistoryBFCL/simple_python_249 (religion.history_info): religion=Christianity, till_century=14, include_people omitted or false.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 50

Failed

IN385 927 OUT4 863

×

SimpleEinsteinContributionBFCL/simple_python_247 (historical_contrib.get_contrib): scientist=Albert Einstein, date=1915-03-17 (variants), category omitted or all.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN386 099 OUT4 446

×

SimpleHistoricalGdpBFCL/simple_python_237 (get_historical_GDP): country=United States/US, start_year=1960, end_year=2000.1 AI query 15 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN76 262 OUT1 553

×

SimpleMriReportBFCL/simple_python_218 (patient.get_mri_report): patient_id=546382, mri_type omitted or brain, status=concluded.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 204 OUT296

×

SimpleNeuronalActivityBFCL/simple_python_220 (calculate_neuronal_activity): input_synaptic_rate=200, weight=0.5, decay_rate=0.1.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 224 OUT345

×

SimpleReadingPreferenceBFCL/simple_python_225 (psych_research.get_preference): category=reading, option_one=digital[reading], option_two=physical book(s), demographic omitted or all.1 AI query 50 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN391 136 OUT4 794

×

SimpleRelativityInventionBFCL/simple_python_248 (science_history.get_invention): invention_name=theory of relativity, want_year=true.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 515 OUT436

×

SimpleRenewableEmissionSavingsBFCL/simple_python_202 (calculate_emission_savings): energy_type=renewable, usage_duration=3, region=California/CA. EXPECTED MODEL MISS (kept, do not loosen): the prompt says "renewable energy sources" generically; models often over-specify a concrete type (e.g. "solar"). Ground truth wants "renewable".1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN30 986 OUT1 507

×

SimpleZodiacCompatibilityBFCL/simple_python_226 (get_zodiac_compatibility): sign1=Aries, sign2=Gemini, scale omitted or percentage.1 AI query 26 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 26

Failed

IN151 730 OUT2 221

Tool Calling - Simple: Law, Finance & Records (set 1) 10 failed

0.0%

IN1 087 190 OUT18 543

×

SimpleAirQualityByDateBFCL/simple_python_196 (air_quality): location London, date reformatted to 08-16-2022.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 111 OUT775

×

SimpleCivilCaseDetailsWithDissentBFCL/simple_python_167 (law.civil.get_case_details): case_title Roe v. Wade, include_dissent true.1 AI query 38 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 38

Failed

IN258 623 OUT4 149

×

SimpleCompoundedInterestQuarterlyBFCL/simple_python_153 (calculate_compounded_interest): principal 5000, rate 0.03, time 5, n 4 (quarterly).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 953 OUT417

×

SimpleCrimeRateByCityBFCL/simple_python_164 (get_crime_rate): city San Francisco, state California/CA, type violent/omitted, year 2020.1 AI query 41 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 41

Failed

IN292 517 OUT3 906

×

SimpleCriminalRecordsByPersonBFCL/simple_python_158 (get_criminal_records): name Mr. X, location New York, NY, from_year 2012, to_year 2015.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN28 424 OUT624

×

SimpleDetailedWeatherForecastHoursBFCL/simple_python_185 (detailed_weather_forecast): location New York, duration 72 hours, include_precipitation true.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 112 OUT335

×

SimpleFutureValueByDurationBFCL/simple_python_155 (calculate_future_value): initial_investment 1000, interest_rate 0.05, duration 2, compounded omitted or 1.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 331 OUT446

×

SimpleLawsuitDetailsWithExtraFieldsBFCL/simple_python_178 (get_lawsuit_details): case_number LAX2019080202, court_location Los Angeles, additional_details omitted or full default list.1 AI query 27 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 27

Failed

IN160 520 OUT2 404

×

SimpleLawsuitSearchByFilingDateBFCL/simple_python_168 (lawsuit_search): company Google/GOOG, start_date 01-01-2021, location California, status ongoing/omitted.1 AI query 40 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 40

Failed

IN273 863 OUT3 636

×

SimpleLawyerExperienceMonthsBFCL/simple_python_175 (lawyer.get_experience): name John Doe, law_type Bankruptcy.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 736 OUT1 851

Tool Calling - Simple: Law, Finance & Records (set 2) 10 failed

0.0%

IN1 309 564 OUT19 501

×

SimpleAirQualityIndexByTimeBFCL/simple_python_197 (get_air_quality_index): location San Diego, time 12pm/12:00.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN22 757 OUT317

×

SimpleCivilCasesRetrieveTheftBFCL/simple_python_165 (civil_cases.retrieve): year 2020, crime_type theft, location Los Angeles.1 AI query 45 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 45

Failed

IN327 700 OUT4 098

×

SimpleFindNearbyDivorceLawyerBFCL/simple_python_166 (lawyer.find_nearby): city Chicago IL, specialty [Divorce], fee 400. NOTE: single-call contract kept intentionally; the model occasionally emits a spurious second call (no other visible tool fits this query), which is a model-reliability signal, not a test defect.1 AI query 15 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN87 146 OUT1 839

×

SimpleGenerateLawContractRentalBFCL/simple_python_162 (generate_law_contract): parties [John, Alice], contract_type Rental Agreement, location California/CA.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN54 043 OUT1 727

×

SimpleLawsuitDetailsFindByCompanyBFCL/simple_python_176 (lawsuit_details.find): company_name Apple Inc., year 2010, case_type Patent/IPR.1 AI query 19 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 19

Failed

IN99 096 OUT1 788

×

SimpleLawsuitDetailsWithVerdictBFCL/simple_python_181 (get_lawsuit_details): case_number 123456-ABC, court_location Los Angeles, with_verdict true.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN32 048 OUT629

×

SimpleLocalNurseryAnnualPlantsBFCL/simple_python_193 (local_nursery.find): location Toronto, plant_types [Annual].1 AI query 44 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 44

Failed

IN303 776 OUT3 556

×

SimplePropertyRecordsWithOwnerBFCL/simple_python_163 (property_records.get): address 123 main street, parcel_number 1234567890, county Santa Clara, include_owner true.1 AI query 15 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN76 214 OUT1 672

×

SimpleRoiSavingsAccountBFCL/simple_python_150 (calculate_roi): deposit 1000, annual_interest_rate 0.03, years 1.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN46 538 OUT991

×

SimpleWeatherForecastDetailedDaysBFCL/simple_python_189 (weather_forecast_detailed): location New York, days 3, details true.1 AI query 39 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 39

Failed

IN260 246 OUT2 884

Tool Calling - Simple: Law, Finance & Records (set 3) 10 failed

0.0%

IN2 372 923 OUT32 024

×

SimpleCaseInfoVictimBFCL/simple_python_160 (get_case_info): docket 2022/AL2562, court California/CA, info_type victim.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN393 696 OUT5 028

×

SimpleCrimeStatuteTheftBFCL/simple_python_161 (crime_statute_lookup): jurisdiction California/CA, crime theft, detail_level detailed.1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN80 633 OUT2 296

×

SimpleCurrentWeatherCelsiusBFCL/simple_python_186 (current_weather_condition): city Tokyo, country Japan, measurement c/omitted.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN376 943 OUT4 164

×

SimpleEnvironmentalAirQualityDaysBFCL/simple_python_199 (environmental_data.air_quality_index): location San Jose, days 3.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 866 OUT329

×

SimpleFelonyCrimeRecordBFCL/simple_python_156 (crime_record.get_record): case_number CA123456, county San Diego, details true.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN385 145 OUT4 666

×

SimpleHighestGrossingBankBFCL/simple_python_151 (highest_grossing_banks): country U.S variants, year 2020, top_n 1.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN392 565 OUT4 993

×

SimpleHistoricalFraudCaseBFCL/simple_python_170 (law_case_search.find_historical): subject fraud, from_year 2010, to_year 2015.1 AI query 23 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 23

Failed

IN128 768 OUT2 233

×

SimpleLawsuitCasesByCompanyYearBFCL/simple_python_177 (get_lawsuit_cases): company_name Facebook, year 2018, status all/omitted. EXPECTED MODEL MISS (kept, do not loosen): "Patent" is the case TYPE, not the status (enum is open/closed/all); status should be omitted/"all". Models often wrongly set status="Patent".1 AI query 30 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 30

Failed

IN181 593 OUT2 955

×

SimpleLawsuitInfoByCaseNumberBFCL/simple_python_182 (lawsuit_info): case_number XYZ123, year omitted/2023, location omitted/all.1 AI query 35 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 35

Failed

IN220 772 OUT2 877

×

SimplePlantsForSlopeErosionBFCL/simple_python_194 (get_plants_for_slope): slope_type hill/steep/moderate, num_results 3.1 AI query 32 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 32

Failed

IN193 942 OUT2 483

Tool Calling - Simple: Law, Finance & Records (set 4) 10 failed

0.0%

IN1 489 629 OUT19 666

×

SimpleActDetailsAmendmentBFCL/simple_python_159 (get_act_details): act_name Criminal Law Amendment Act, amendment_year 2013.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN380 768 OUT3 797

×

SimpleCheckFeloniesByNameBFCL/simple_python_157 (criminal_history.check_felonies): full_name John Doe, birth_date 01-01-1980, state California/CA.1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN83 523 OUT1 569

×

SimpleCourtCaseSearchNoFullTextBFCL/simple_python_169 (court_case.search): docket_number 123456, location Texas, full_text false/omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 733 OUT428

×

SimpleCurrentWeatherTempHumidityBFCL/simple_python_187 (get_current_weather): location Seattle, include_temperature true/omitted, include_humidity true/omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 133 OUT160

×

SimpleLawsuitCheckClosedStatusBFCL/simple_python_184 (lawsuit.check_case): case_id 1234, closed_status true.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 178 OUT759

×

SimpleLawsuitsSearchByCompanyLocationBFCL/simple_python_180 (lawsuits_search): company_name Google, location California/CA, year 2020, case_type all/omitted.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN373 671 OUT4 154

×

SimpleMutualFundBalanceBFCL/simple_python_152 (calculate_mutual_fund_balance): investment_amount 50000, annual_yield 0.05, years 3.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 246 OUT454

×

SimpleSlopeGradientCoordinatesBFCL/simple_python_192 (calculate_slope_gradient): point1 [40.7128,-74.006], point2 [34.0522,-118.2437], unit degree/omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 486 OUT326

×

SimpleStateLawCaseYearRangeBFCL/simple_python_173 (law_case_search): topic land disputes, year_range [2015,2021], location New York, judicial_system state.1 AI query 30 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 30

Failed

IN186 747 OUT2 954

×

SimpleTopConstitutionalCasesBFCL/simple_python_174 (get_top_cases): field_of_law constitutional law, top_number 10, country China/CN.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 50

Failed

IN388 144 OUT5 065

Tool Calling - Simple: Law, Finance & Records (set 5) 10 failed

0.0%

IN1 863 732 OUT25 216

×

SimpleCarbonFootprintLifestyleBFCL/simple_python_195 (calculate_carbon_footprint): daily_miles 20, meat_meals_per_week 3, annual_trash_weight 500, flights_per_year omitted/0.1 AI query 43 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 43

Failed

IN299 916 OUT3 905

×

SimpleDailyWaterIntakeWeightBFCL/simple_python_198 (calculate_daily_water_intake): weight 70, activity_level omitted/moderate, climate omitted/temperate.1 AI query 34 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 34

Failed

IN209 041 OUT2 821

×

SimpleFetchLawCaseByNumberBFCL/simple_python_171 (fetch_law_case_details): case_number 43403, court New York, year 2018.1 AI query 23 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 23

Failed

IN125 838 OUT2 028

×

SimpleFutureValueByPresentValueBFCL/simple_python_154 (calculate_future_value): present_value 5000, annual_interest_rate 0.05, years 10, compounds_per_year omitted/1.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN372 646 OUT4 145

×

SimpleHumidityForecastDaysBFCL/simple_python_188 (weather.humidity_forecast): location Miami, days 7, min_humidity omitted/0.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN36 756 OUT789

×

SimpleLatestCourtCaseCompaniesBFCL/simple_python_179 (find_latest_court_case): company1 Apple, company2 Samsung, country USA/omitted.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN383 372 OUT4 598

×

SimpleLawsuitSearchByCountyBFCL/simple_python_183 (lawsuit_search): entity Apple, county Santa Clara County, state California/omitted.1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 18

Failed

IN92 327 OUT1 391

×

SimpleLegalCaseFetchDetailsBFCL/simple_python_172 (legal_case.fetch): case_id R vs Adams, details true.1 AI query 23 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 23

Failed

IN125 366 OUT2 076

×

SimpleParkElevationAreaBFCL/simple_python_190 (park_information): park_name Yellowstone, information [Elevation, Area] (order-independent).1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 174 OUT481

×

SimpleTallestMountainsRadiusBFCL/simple_python_191 (locate_tallest_mountains): location Denver, radius 50, amount 5.1 AI query 32 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 32

Failed

IN195 296 OUT2 982

Tool Calling - Simple: Live Answer Tool (isolated) (set 1) 1 failed

0.0%

IN280 429 OUT3 504

×

SimpleAnswerStringLogistic1BFCL/live_simple_32-9-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L113: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN280 429 OUT3 504

Tool Calling - Simple: Live Answer Tool (isolated) (set 2) 1 passed

100.0%

IN3 980 OUT96

✓

SimpleAnswerStringLogistic2BFCL/live_simple_33-10-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L114: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN3 980 OUT96

Tool Calling - Simple: Live Answer Tool (isolated) (set 3) 1 skipped

-

IN0 OUT0

–

SimpleAnswerStringValidBFCL/live_simple_34-11-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L115: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Harness-semantic mismatch (item flaw for our runner): the query ('...Valid?') is directly answerable, and our system prompt permits answering in plain text when no tool is needed, so the model reasonably returns 0 tool calls instead of routing to the answer_string meta-tool. Not a deterministic tool-routing test. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Simple: Live Answer Tool (isolated) (set 4) 1 failed

0.0%

IN563 768 OUT14 987

×

SimpleAnswerStringWhatIsBFCL/live_simple_35-12-0 (answer.string): answer states logistic regression is not present / cannot be answered. ISOLATED from ToolCalling-Simple-L116: answer_string is a universal "respond to the user" collider (it can answer ANY query), so alongside specific tools it makes the 27B oscillate (Completed=false loops that broke the co-located tests). Solo file => only this tool is visible.1 AI query 50 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN563 768 OUT14 987

Tool Calling - Simple: Live Assistants, Weather & Devices (set 1) 1 passed2 failed

33.3%

IN35 411 OUT783

✓

SimpleChangeFoodBurgerBFCL/live_simple_20-4-0 (change_food): food_item='burger', modification_request='no onions, extra cheese'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 903 OUT88

×

SimpleThinqAirCleanHighBFCL/live_simple_40-17-0 (ThinQ_Connect): AIR_CLEAN mode, HIGH wind, monitoring on, air clean POWER_ON. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. windStrength broadened (high|strongest|strong) for "strongest wind strength". EXPECTED MODEL MISS: 27B sets airConJobMode=AIR_CLEAN but omits airCleanOperationMode=POWER_ON — the prompt asks to "start the air purifying function", so POWER_ON is the fair GT; kept strict (a real tool-use miss).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 572 OUT129

×

SimpleWeatherTelAvivBFCL/live_simple_4-3-0 (get_current_weather): location='Tel Aviv, Israel', unit=fahrenheit or omitted.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN22 936 OUT566

Tool Calling - Simple: Live Assistants, Weather & Devices (set 10) 1 passed2 failed

33.3%

IN88 518 OUT1 777

×

SimpleChafodOrderPizzaBFCL/live_simple_23-5-1 (ChaFod): TheFod='PIZZA'.1 AI query 8 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN25 338 OUT627

✓

SimpleUberEatMcdonaldsPizzaBFCL/live_simple_29-7-2 (uber.eat.order): restaurant="McDonald's", items=[pizza], quantities=[1].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 664 OUT97

×

SimpleWeatherShanghaiBFCL/live_simple_13-3-9 (get_current_weather): location='Shanghai, China', unit=celsius. Prompt is Chinese, copied verbatim.1 AI query 16 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN58 516 OUT1 053

Tool Calling - Simple: Live Assistants, Weather & Devices (set 11) 1 passed2 failed

33.3%

IN75 237 OUT1 893

✓

SimpleChafodSwitchBurgerBFCL/live_simple_24-5-2 (ChaFod): TheFod='BURGER'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 009 OUT64

×

SimpleFindBeerSierraNevadaBFCL/live_simple_48-21-0 (find_beer): brewery='Sierra Nevada', taste=bitter, aroma=hoppy, color=pale, style=lager; defaults omitted.1 AI query 13 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 13

Failed

IN50 179 OUT1 438

×

SimpleWeatherChennaiBFCL/live_simple_15-3-11 (get_current_weather): location='Chennai, Tamil Nadu', unit=fahrenheit or omitted.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN20 049 OUT391

Tool Calling - Simple: Live Assistants, Weather & Devices (set 12) 3 failed

0.0%

IN383 510 OUT5 736

×

SimpleChafodGreensSaladBFCL/live_simple_25-5-3 (ChaFod): TheFod='SALAD' (greens mapped to salad).1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN314 849 OUT3 987

×

SimpleFindBeerCreekPorterBFCL/live_simple_49-21-1 (find_beer): brewery='Creek', taste='slightly bitter', aroma=chocolatey, color=dark, style=porter; defaults omitted.1 AI query 15 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN60 883 OUT1 605

×

SimpleWeatherQuintanaRooBFCL/live_simple_19-3-15 (get_current_weather): location='Quintana Roo, Mexico', unit=celsius.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 778 OUT144

Tool Calling - Simple: Live Assistants, Weather & Devices (set 13) 1 passed1 failed

50.0%

IN12 173 OUT346

✓

SimpleLexListExportsV2BFCL/live_simple_30-8-0 (aws.lexv2_models.list_exports): botId='my-bot-id', botVersion='v2', sortBy=ASC or omitted, others omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 842 OUT176

×

SimpleWeatherLangSonBFCL/live_simple_16-3-12 (get_current_weather): location='Lang Son, Vietnam', unit=fahrenheit or omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 331 OUT170

Tool Calling - Simple: Live Assistants, Weather & Devices (set 14) 1 passed1 failed

50.0%

IN17 732 OUT388

✓

SimpleLexListExportsDescBFCL/live_simple_31-8-1 (aws.lexv2_models.list_exports): botId='B12345', botVersion='v1', sortBy=DESC, others omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 846 OUT107

×

SimpleWeatherBostonBFCL/live_simple_17-3-13 (get_current_weather): location='Boston, MA', unit=fahrenheit or omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 886 OUT281

Tool Calling - Simple: Live Assistants, Weather & Devices (set 15) 2 failed

0.0%

IN23 921 OUT603

×

SimpleGetUserInfoBlackBFCL/live_simple_0-0-0 (get_user_info): user_id=7890, special='black'.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN9 234 OUT240

×

SimpleWeatherMoscowBFCL/live_simple_18-3-14 (get_current_weather): location='Moscow, Russia', unit=celsius.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN14 687 OUT363

Tool Calling - Simple: Live Assistants, Weather & Devices (set 16) 3 failed

0.0%

IN37 019 OUT1 072

×

SimpleFetchWeatherCoordsBFCL/live_simple_39-16-0 (fetch_weather_data): latitude=37.8651, longitude=-119.5383; url/units/language defaults omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 607 OUT309

×

SimpleGithubStarGorillaBFCL/live_simple_1-1-0 (github_star): repos='ShishirPatil/gorilla,gorilla-llm/gorilla-cli', aligned=true.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 261 OUT412

×

SimpleWeatherBangkokBFCL/live_simple_14-3-10 (get_current_weather): location='Bangkok, Thailand', unit=fahrenheit or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN16 151 OUT351

Tool Calling - Simple: Live Assistants, Weather & Devices (set 17) 1 passed2 failed

33.3%

IN58 856 OUT1 629

×

SimpleCarbonIntensityGBBFCL/live_simple_50-22-0 (get_latest_carbon_intensity): zone='Great Britain' or 'GB'.1 AI query 13 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 13

Failed

IN44 130 OUT1 011

✓

SimpleMultiplyThreeTwoBFCL/live_simple_47-20-0 (multiply): a=3, b=2.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 596 OUT63

×

SimpleWeatherOpenMeteoMariposaBFCL/live_simple_38-15-0 (get_current_weather, open-meteo variant): location='Mariposa, CA', unit=fahrenheit; url default omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 130 OUT555

Tool Calling - Simple: Live Assistants, Weather & Devices (set 2) 2 passed1 failed

66.7%

IN327 228 OUT4 099

✓

SimpleChangeFoodChickenSpicyBFCL/live_simple_21-4-1 (change_food): food_item='chicken dish', modification_request='extra spicy'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 920 OUT69

✓

SimpleThinqAirDryMidBFCL/live_simple_41-17-1 (ThinQ_Connect): AIR_DRY mode, MID wind (or omitted), all else optional. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. windStrength broadened (mid|medium) for "medium wind strength"; air-clean off-state accepts off/power_off.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 893 OUT79

×

SimpleWeatherDivinopolisBFCL/live_simple_5-3-1 (get_current_weather): location='Divinopolis, MG' (verbatim), unit=fahrenheit or omitted. Prompt is Portuguese, copied verbatim.1 AI query 50 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN317 415 OUT3 951

Tool Calling - Simple: Live Assistants, Weather & Devices (set 3) 1 passed2 failed

33.3%

IN48 134 OUT992

×

SimpleParseAnswerNoDefineBFCL/live_simple_36-13-0 (parseAnswer): answer='The text does not define logistic regression, therefore I cannot answer this question.'1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN29 995 OUT594

✓

SimpleThinqKoreanTarget22BFCL/live_simple_42-17-2 (ThinQ_Connect): coolTargetTemperature=22 required; mode COOL and wind MID optional. Prompt is Korean, copied verbatim. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. Prompt is dehumidify (제습) so GT's optional airConJobMode=COOL is a source oddity — accept cool/dehumidify/air_dry (optional). windStrength MID broadened (mid|medium). The "22도" target-temp is accepted in either coolTargetTemperature or targetTemperature (the two are genuinely ambiguous for "target temperature").1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 924 OUT132

×

SimpleWeatherSanFranciscoBFCL/live_simple_6-3-2 (get_current_weather): location='San Francisco, CA', unit=fahrenheit or omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 215 OUT266

Tool Calling - Simple: Live Assistants, Weather & Devices (set 4) 1 passed2 failed

33.3%

IN217 610 OUT3 660

×

SimpleParseAnswerNoDefineValidityBFCL/live_simple_37-14-0 (parseAnswer): answer='The text does not define logistic regression, therefore I cannot answer this question.'1 AI query 36 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 36

Failed

IN196 549 OUT3 238

✓

SimpleThinqCoolHighWindBFCL/live_simple_43-17-3 (ThinQ_Connect): HIGH wind required; COOL mode and coolTargetTemperature=24 optional. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. air-clean off-state broadened (power_off|off) — the model files the unrequested clean-mode as "off".1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 872 OUT89

×

SimpleWeatherRigaBFCL/live_simple_7-3-3 (get_current_weather): location='Riga, Latvia'; unit fahrenheit/celsius/omitted all accepted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN16 189 OUT333

Tool Calling - Simple: Live Assistants, Weather & Devices (set 5) 1 passed2 failed

33.3%

IN17 911 OUT434

×

SimpleThinqTurnedOnCoolBFCL/live_simple_44-18-0 (ThinQ_Connect): airConOperationMode=POWER_ON required; COOL currentJobMode optional. NOTE: tool flattened to nullable top-level params (see struct comment) so a valid partial call is one tool call, not a forced retry. windStrength/airClean off-state broadened to synonyms (mid|medium, stop|off|power_off).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.coolTargetTemperature.AnyOf([null, 24]) Values: args.coolTargetTemperature = 75

Failed

IN5 056 OUT87

✓

SimpleUberComfortAddisonBFCL/live_simple_2-2-0 (uber.ride): loc='2020 Addison Street, Berkeley, CA, USA', type=comfort, time=600.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 107 OUT176

×

SimpleWeatherLondonBFCL/live_simple_8-3-4 (get_current_weather): location='London, UK' or 'London, England'; unit fahrenheit or omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 748 OUT171

Tool Calling - Simple: Live Assistants, Weather & Devices (set 6) 1 passed2 failed

33.3%

IN20 689 OUT480

✓

SimpleThinqCoolAirCleanStartBFCL/live_simple_45-18-1 (ThinQ_Connect): airCleanOperationMode=START, airConOperationMode=POWER_ON, coolTargetTemperature=20. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. airCleanOperationMode "air cleaning on" broadened (start|on|power_on). airConOperationMode=POWER_ON and coolTargetTemperature=20 kept strict — both fair GT. EXPECTED MODEL MISS: 27B gets job-mode + air-clean START + temp, but omits airConOperationMode=POWER_ON.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 109 OUT105

×

SimpleUberPlusBakerBFCL/live_simple_3-2-1 (uber.ride): loc='221B Baker Street, Berkeley, CA, USA', type=plus, time=600.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 859 OUT194

×

SimpleWeatherHyderabadBFCL/live_simple_9-3-5 (get_current_weather): location='Hyderabad, India', unit=fahrenheit or omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 721 OUT181

Tool Calling - Simple: Live Assistants, Weather & Devices (set 7) 1 passed2 failed

33.3%

IN26 541 OUT904

✓

SimpleThinqFlatCoolCleanBFCL/live_simple_46-19-0 (ThinQ_Connect, flat params): COOL, MID, monitoring on, clean START, POWER_ON, power-save off, target 24C.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 148 OUT198

×

SimpleUberPlusVietnameseBFCL/live_simple_26-6-0 (uber.ride): loc Vietnamese/English form, type=plus, time=10 (minutes). Prompt is Vietnamese, copied verbatim.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 808 OUT213

×

SimpleWeatherYosemiteMariposaBFCL/live_simple_10-3-6 (get_current_weather): location='Mariposa, CA', unit=celsius.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 585 OUT493

Tool Calling - Simple: Live Assistants, Weather & Devices (set 8) 3 failed

0.0%

IN29 112 OUT809

×

SimpleThinqCoolScheduledBFCL/live_simple_51-23-0 (ThinQ_Connect): COOL, HIGH wind, POWER_ON, power-save on, target 22C, start in 1h30m. NOTE: tool flattened to nullable top-level params (see struct comment) to avoid forced-retry Count>1. airConOperationMode=POWER_ON kept strict ("Start in 1h30m" schedules a power-on). EXPECTED MODEL MISS: 27B gets job-mode COOL + wind + power-save + temp + schedule, but omits airConOperationMode=POWER_ON.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: (args.airConOperationMode?.ToLower()).AnyOf(["power_on"])

Failed

IN5 100 OUT178

×

SimpleUberEatPitadaBFCL/live_simple_27-7-0 (uber.eat.order): restaurant='uber pitada', items=[burgers, chicken wings], quantities=[5, 6].1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 586 OUT255

×

SimpleWeatherNaplesBFCL/live_simple_11-3-7 (get_current_weather): location='Naples, FL' or 'Naples, Florida'; unit fahrenheit or omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 426 OUT376

Tool Calling - Simple: Live Assistants, Weather & Devices (set 9) 3 failed

0.0%

IN84 276 OUT1 882

×

SimpleChafodWhopperBurgerBFCL/live_simple_22-5-0 (ChaFod): TheFod='BURGER' (Whopper mapped to burger).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 753 OUT271

×

SimpleUberEatKfcChineseBFCL/live_simple_28-7-1 (uber.eat.order): KFC order, 4 items, quantities [10,50,30,90]. Prompt is Chinese, copied verbatim.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN16 429 OUT552

×

SimpleWeatherNewYorkBFCL/live_simple_12-3-8 (get_current_weather): location='New York, NY', unit=celsius.1 AI query 15 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN54 094 OUT1 059

Tool Calling - Simple: Live Media, Weather & Data (set 2) 4 passed6 failed

40.0%

IN313 592 OUT5 108

×

SimpleCreditQuoteAutoCreditoBFCL/live_simple_67-31-0 (obtener_cotizacion_de_creditos): monto=1000000, plazo=12, producto='auto', año=2024, enganche=0.2.1 AI query 29 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 29

Failed

IN199 976 OUT3 237

×

SimpleCurrentLocDmsBFCL/live_simple_87-48-0 (get_current_loc): coordinate_format='DMS', include_altitude=true, timeout default/omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 878 OUT379

×

SimpleGetMoviesMumbaiTodayBFCL/live_simple_59-28-0 (get_movies): city='Mumbai', optionals default/omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 149 OUT337

✓

SimplePlayArtistKaarijaTracksBFCL/live_simple_93-54-0 (play_artist): artist_name='Käärijä'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 570 OUT62

✓

SimpleRetrieveAnalysisSuperfancyBFCL/live_simple_69-33-0 (analysis_api.AnalysisApi.retrieve_analysis): project/component/vulnerability UUIDs.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 685 OUT235

×

SimpleSensorAlertsPerPage10BFCL/live_simple_70-34-0 (get_sensor_alerts): perPage=10, sensorSerial='Q3CC-CRT3-SZ2G', rest omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 802 OUT368

✓

SimpleSitefinityNewsItemJulian12BFCL/live_simple_81-42-0 (sitefinity_create_contentitem): ContentItem='NewsItem', Title='Julian is testing12', rest omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 636 OUT84

✓

SimpleTodoAddShopping9pmB25BFCL/live_simple_55-25-1 (todo): type='add', content='go for shopping at 9 pm'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 579 OUT75

×

SimpleTodoDeleteTodoRandomBFCL/live_simple_54-25-0 (todo): type='delete', content='todo random'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.content?.ToLower()).AnyOf(["todo random"])

Failed

IN7 581 OUT81

×

SimpleWeatherGetLondonUkBFCL/live_simple_77-38-0 (weather.get): city='London', country='GB' (from UK), units/forecast default/omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 736 OUT250

Tool Calling - Simple: Live Media, Weather & Data (set 3) 6 passed4 failed

60.0%

IN528 630 OUT8 579

×

SimpleExtractMillennialsAppleBFCL/live_simple_71-35-0 (extract_parameters_v1): demographics=[millennials], targets=[brand:Apple], metrics=[view], min_date=2022-07-01. NOTE: targets @param clarified to '<attribute>:<entity>' order (was ambiguous 'target:target_type', which made the model emit the reversed 'Apple:brand'); GT 'brand:Apple' kept strict.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN60 985 OUT1 966

✓

SimpleFetchPersonId4BFCL/live_simple_91-52-0 (fetchPerson): id=4.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 141 OUT829

×

SimpleLogFoodChaiTeaBFCL/live_simple_88-49-0 (log_food): food_name='chai tea', portion_amount=16.0, portion_unit='ounces', meal_name='snack'.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN399 019 OUT5 017

✓

SimplePlaySongChaChaChaBFCL/live_simple_94-55-0 (play_song): artist_name='Käärijä', song_name='Cha Cha Cha'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 159 OUT76

✓

SimpleSendEmailAndyForecastBFCL/live_simple_78-39-0 (send_email): to='andy@gorilla.ai', subject='Sales Forecast Request', body set, cc/bcc/attachments omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 207 OUT109

✓

SimpleSitefinityNewsJulianTestingBFCL/live_simple_82-43-0 (sitefinity_create_contentitem): ContentItem='News', Title='Julian is testing', rest omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 175 OUT79

×

SimpleTemperatureParisFranceBFCL/live_simple_90-51-0 (temperature): location='Paris, France', units/time default/omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 722 OUT187

✓

SimpleTodoManagerAddSleepBFCL/live_simple_57-26-1 (todo_manager.handle_action): type='add', content='go to sleep at 9 pm'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 146 OUT77

✓

SimpleTodoManagerDeleteGymBFCL/live_simple_56-26-0 (todo_manager.handle_action): type='delete', content='go to gym'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 136 OUT70

×

SimpleTranslateEnglishFrenchBFCL/live_simple_76-37-0 (language_translator.translate): source='en', target='fr', text='What is your name?'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 940 OUT169

Tool Calling - Simple: Live Media, Weather & Data (set 4) 5 passed5 failed

50.0%

IN453 923 OUT6 460

×

SimpleOpenMeteoBerkeleyBFCL/live_simple_85-46-0 (open_meteo_api.fetch_weather_data): coordinates=[37.8715,-122.273], units default/omitted.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN385 192 OUT5 102

×

SimpleSearchHotelsSanDiegoBFCL/live_simple_79-40-0 (search_hotels): location='San Diego, CA'.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN17 684 OUT431

✓

SimpleSitefinityNewsSleepProductivityBFCL/live_simple_83-44-0 (sitefinity_create_contentitem): full news article with all metadata.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 482 OUT244

✓

SimpleSortArrayAscendingBFCL/live_simple_86-47-0 (sort_array): array=[1,54,3,1,76,2343,21], reverse default/omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 358 OUT105

×

SimpleTodoAddGoToGoaBFCL/live_simple_64-29-4 (todo): type='add', content='go to Goa'.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN8 096 OUT119

✓

SimpleTodoAddGymTomorrowBFCL/live_simple_61-29-1 (todo): type='add', content='go to gym tomorrow.'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 284 OUT69

✓

SimpleTodoAddHiCharlieBFCL/live_simple_65-29-5 (todo): type='add', content='Hi charlie'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 295 OUT69

✓

SimpleTodoAddSleep9pmBFCL/live_simple_63-29-3 (todo): type='add', content='go to sleep at 9 pm'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 291 OUT75

×

SimpleTodoCompleteFinancialReportBFCL/live_simple_60-29-0 (todo): type='complete', content='Submit monthly financial report'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN8 132 OUT128

×

SimpleTodoDeleteRaviBFCL/live_simple_62-29-2 (todo): type='delete', content='ravi'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN8 109 OUT118

Tool Calling - Simple: Live Movies, Music & Booking (set 1) 2 passed1 failed

66.7%

IN40 345 OUT972

×

SimpleMoviesTarantinoWhitakerBFCL/live_simple_208-117-0 (Movies_3_FindMovies): directed_by='Quentin Tarantino', cast='Duane Whitaker', genre dontcare.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN30 626 OUT830

✓

SimplePlaySongRoarBFCL/live_simple_230-121-0 (play_song): song_id='wjeiruhuq_roar', artist='Katy Perry', quality='high', device_id omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 887 OUT86

✓

SimpleSetVolume20BFCL/live_simple_242-126-0 (set_volume): volume=20.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 832 OUT56

Tool Calling - Simple: Live Movies, Music & Booking (set 10) 2 passed1 failed

66.7%

IN349 080 OUT4 779

✓

SimpleHandoverPackageArrivalBFCL/live_simple_257-137-1 (answer_question): statement=package-arrival-time question, urgency/language default. System instructs HANDOVER on question. NOTE: no collider loop observed (single clean call, sibling tests pass); statement is free text -> substring-intent of the key phrase (model drops "Can you tell me").1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 221 OUT107

×

SimpleMoviesNiteshDirectorBFCL/live_simple_217-117-9 (Movies_3_FindMovies): directed_by='Nitesh Tiwari', genre/cast dontcare.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN338 642 OUT4 561

✓

SimpleTtsChineseMaleBFCL/live_simple_227-118-1 (text_to_speech.convert): text=Chinese phrase, language='zh-CN', gender='male', format='wav', speed default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 217 OUT111

Tool Calling - Simple: Live Movies, Music & Booking (set 11) 2 passed1 failed

66.7%

IN406 226 OUT8 270

×

SimpleMoviesColaizzoWatkinsBFCL/live_simple_218-117-10 (Movies_3_FindMovies): directed_by='Paul Downs Colaizzo', cast='Michaela Watkins', genre dontcare.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN396 307 OUT7 742

✓

SimpleRequestsGetBuProductsBFCL/live_simple_229-120-0 (requests.get): anchor='user'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 958 OUT430

✓

SimpleTtsPrettyGirlDefaultBFCL/live_simple_228-119-0 (text_to_speech.convert): text='I am a pretty girl', language/gender/speed default.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 961 OUT98

Tool Calling - Simple: Live Movies, Music & Booking (set 12) 3 failed

0.0%

IN34 788 OUT1 630

×

SimpleBookFlightParisBFCL/live_simple_233-123-0 (book_flight): departure_location='Paris, France', departure_time='12/03/2023 15:00', return_time omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 589 OUT235

×

SimpleGetVersionBFCL/live_simple_247-129-0 (version_api.VersionApi.get_version): no arguments.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 406 OUT412

×

SimpleMoviesSimonCurtisDramaBFCL/live_simple_219-117-11 (Movies_3_FindMovies): directed_by='Simon Curtis', genre='Drama', cast='McKinley Belcher III'.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN19 793 OUT983

Tool Calling - Simple: Live Movies, Music & Booking (set 13) 3 failed

0.0%

IN59 921 OUT2 611

×

SimpleAclRetrieveProjectsBFCL/live_simple_248-130-0 (acl_api.AclApi.retrieve_projects): uuid='12', excludeInactive=true, onlyRoot=true.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 716 OUT595

×

SimpleBookFlightNewYorkToParisBFCL/live_simple_234-123-1 (book_flight): departure_location='New York, NY', departure_time='20/06/2022 17:00', return_time omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 810 OUT291

×

SimpleMoviesMackendrickThrillerBFCL/live_simple_220-117-12 (Movies_3_FindMovies): directed_by='Alexander Mackendrick', genre='Thriller', cast dontcare.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN35 395 OUT1 725

Tool Calling - Simple: Live Movies, Music & Booking (set 14) 1 passed2 failed

33.3%

IN55 921 OUT1 496

×

SimpleBookFlightJfkBFCL/live_simple_245-127-0 (book_flight): departure_location='JFK', departure_date='16/03/2024', departure_time='14:00'.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN20 304 OUT540

✓

SimpleChatWithBunnyBFCL/live_simple_249-131-0 (chat_with_friend): user_message='Hey bunny, How are you doing'. NOTE: no collider loop observed (single clean call, sibling tests pass); user_message is a free-text passthrough -> substring-intent (model may drop the "Hey bunny," greeting).1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 911 OUT77

×

SimpleMoviesJamesGrayBFCL/live_simple_221-117-13 (Movies_3_FindMovies): directed_by='James Gray', genre/cast dontcare.1 AI query 9 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN30 706 OUT879

Tool Calling - Simple: Live Movies, Music & Booking (set 15) 1 passed1 failed

50.0%

IN332 323 OUT5 030

✓

SimpleBookFlightNewYorkMarchBFCL/live_simple_246-128-0 (book_flight): departure_location='New York, NY', departure_time='15/03/2024'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 646 OUT137

×

SimpleMoviesVanOrmanBrownBFCL/live_simple_222-117-14 (Movies_3_FindMovies): directed_by='Thurop Van Orman', cast='Sterling K. Brown', genre dontcare.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN327 677 OUT4 893

Tool Calling - Simple: Live Movies, Music & Booking (set 16) 1 failed1 skipped

0.0%

IN9 918 OUT401

×

SimpleMoviesImaginativeFantasyBFCL/live_simple_223-117-15 (Movies_3_FindMovies): genre='Fantasy' ('Imaginative fiction'), director/cast dontcare.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN9 918 OUT401

–

SimpleRescheduleGymSessionBFCL/live_simple_231-122-0 (reschedule_event): event_identifier='456123', new_datetime='2022-10-30T16:30:00Z'.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: source flaw: relative-date arithmetic is inconsistent — prompt states 'Monday 2022-10-20' (that date is actually a Thursday) and the GT 'next Thursday' resolves to 2022-10-30 (a Sunday), so the exact new_datetime cannot be derived deterministically.

Skipped

IN0 OUT0

Tool Calling - Simple: Live Movies, Music & Booking (set 17) 1 passed1 failed

50.0%

IN17 542 OUT531

×

SimpleMoviesTimBurtonDeppBFCL/live_simple_224-117-16 (Movies_3_FindMovies): directed_by='Tim Burton', genre='Offbeat', cast='Johnny Depp'.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 861 OUT407

✓

SimpleRescheduleMedAppointmentBFCL/live_simple_232-122-1 (reschedule_event): event_identifier='med123', new_datetime='2021-04-15T13:45:00Z'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 681 OUT124

Tool Calling - Simple: Live Movies, Music & Booking (set 18) 1 passed1 failed

50.0%

IN14 216 OUT269

×

SimpleMoviesKirillComedyDramaBFCL/live_simple_225-117-17 (Movies_3_FindMovies): directed_by='Kirill Mikhanovsky', genre='Comedy-drama', cast dontcare.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN9 770 OUT213

✓

SimpleSetVolume70BFCL/live_simple_244-126-2 (set_volume): volume=70.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 446 OUT56

Tool Calling - Simple: Live Movies, Music & Booking (set 2) 2 passed1 failed

66.7%

IN17 261 OUT275

✓

SimpleMoviesLeitchTuisanoBFCL/live_simple_209-117-1 (Movies_3_FindMovies): directed_by='David Leitch', cast='Lori Pelenise Tuisano', genre dontcare.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 915 OUT97

✓

SimpleSetVolume30BFCL/live_simple_243-126-1 (set_volume): volume=30.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 868 OUT58

×

SimpleSpotifyFriendsMarshmelloBFCL/live_simple_235-124-0 (play_spotify_song): query='track:Friends artist:Marshmello', shuffle/volume default.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 478 OUT120

Tool Calling - Simple: Live Movies, Music & Booking (set 3) 1 passed2 failed

33.3%

IN91 851 OUT2 097

×

SimpleFlightStatusIndigo6E123BFCL/live_simple_251-133-0 (flight.status.check): flight_id='6E123', passenger_name='Karan', ticket_number='IND4567'.1 AI query 17 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 17

Failed

IN70 154 OUT1 578

×

SimpleMoviesDramaBFCL/live_simple_210-117-2 (Movies_3_FindMovies): genre='Drama', director/cast dontcare.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN16 640 OUT444

✓

SimpleSpotifyDilNu124BFCL/live_simple_236-124-1 (play_spotify_song): query='track:Dil Nu artist:Maninder Buttar', shuffle/volume default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 057 OUT75

Tool Calling - Simple: Live Movies, Music & Booking (set 4) 3 failed

0.0%

IN62 676 OUT1 420

×

SimpleFlightStatusIndigoAirlineBFCL/live_simple_252-134-0 (flight.status.check): airlines='Indigo', person_name='Karan', ticket_number='IND4567'. NOTE: tool doc says "if only one name mentioned ... use LNU", so "Karan LNU" is equivalent-correct per the documented convention -> accept both.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN31 916 OUT750

×

SimpleMoviesComedyCordenBFCL/live_simple_211-117-3 (Movies_3_FindMovies): genre='Comedy' (prompt 'Comic'), cast='James Corden', director dontcare. NOTE: args are correct; single-call contract stays strict. The 27B occasionally emits a 2nd call (over-call) -> legit intermittent fail (kept as signal).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN20 136 OUT460

×

SimpleSpotifyWreckingBallBFCL/live_simple_237-125-0 (play_spotify_song): query='track:Wrecking Ball artist:Miley Cyrus', device_id/play default. NOTE: GT accepts only the structured 'track:.. artist:..' form (kept strict). The 27B occasionally drops the format (e.g. "wrecking ball Miley Cyrus") -> legit intermittent miss (kept as signal).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 624 OUT210

Tool Calling - Simple: Live Movies, Music & Booking (set 5) 3 failed

0.0%

IN144 594 OUT3 005

×

SimpleMoviesComedyEdgarWrightBFCL/live_simple_212-117-4 (Movies_3_FindMovies): directed_by='Edgar Wright', genre='Comedy', cast dontcare.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 826 OUT477

×

SimpleSpotifyDilNu125BFCL/live_simple_238-125-1 (play_spotify_song): query='track:Dil Nu artist:Maninder Buttar', device_id/play default.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 742 OUT141

×

SimpleUberEatsBigMacBFCL/live_simple_250-132-0 (uber.eat.order): restaurants='McDonald's', items=[Big Mac, McFlurry], quantities=[2,1].1 AI query 26 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 26

Failed

IN126 026 OUT2 387

Tool Calling - Simple: Live Movies, Music & Booking (set 6) 1 passed2 failed

33.3%

IN100 994 OUT2 138

×

SimpleMoviesOffbeatTimBurtonBFCL/live_simple_213-117-5 (Movies_3_FindMovies): directed_by='Tim Burton', genre='Offbeat', cast dontcare.1 AI query 21 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 21

Failed

IN88 193 OUT1 901

×

SimpleSpotifyShapeOfYouBFCL/live_simple_239-125-2 (play_spotify_song): query='track:Shape of You artist:Ed Sheeran' or 'Shape of You'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 730 OUT124

✓

SimpleUberEatsBurgersWingsBFCL/live_simple_253-135-0 (uber.eat.order): restaurants='McDonald's', items=[burgers, chicken wings], quantities=[5,6].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 071 OUT113

Tool Calling - Simple: Live Movies, Music & Booking (set 7) 3 failed

0.0%

IN43 994 OUT976

×

SimpleMoviesComicNiteshTiwariBFCL/live_simple_214-117-6 (Movies_3_FindMovies): directed_by='Nitesh Tiwari', genre='Comedy' or 'Comedy-drama', cast dontcare.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 701 OUT401

×

SimpleSpotifyBabySharkBFCL/live_simple_240-125-3 (play_spotify_song): query='Baby Shark' (artist unknown).1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 420 OUT197

×

SimpleUserMandatesPendingBFCL/live_simple_254-136-0 (user.mandates): user_id='U123', status='pending'.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN19 873 OUT378

Tool Calling - Simple: Live Movies, Music & Booking (set 8) 3 failed

0.0%

IN343 857 OUT4 638

×

SimpleMoviesFantasyTonightBFCL/live_simple_215-117-7 (Movies_3_FindMovies): genre='Fantasy', director/cast dontcare.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN328 595 OUT4 217

×

SimpleSpotifyJohnnyPapaBFCL/live_simple_241-125-4 (play_spotify_song): query='Johnny Johnny Yes Papa' (artist unknown).1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 595 OUT126

×

SimpleUserMandatesActiveBFCL/live_simple_255-136-1 (user.mandates): user_id='U123456', status='active' or omitted (default).1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 667 OUT295

Tool Calling - Simple: Live Movies, Music & Booking (set 9) 1 passed2 failed

33.3%

IN24 380 OUT582

×

SimpleHandoverComplaintStatusBFCL/live_simple_256-137-0 (answer_question): statement=user complaint-status question, urgency/language default. System instructs HANDOVER on question.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN11 246 OUT267

×

SimpleMoviesActionLeitchKingBFCL/live_simple_216-117-8 (Movies_3_FindMovies): directed_by='David Leitch', genre='Action', cast='Alex King'.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 947 OUT153

✓

SimpleTtsPrettyGirlSpanishBFCL/live_simple_226-118-0 (text_to_speech.convert): text='I am a pretty girl' or Spanish translation, language='es-ES', gender/format/speed default.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 187 OUT162

Tool Calling - Simple: Live Services, Weather & Commands (set 1) 4 failed

0.0%

IN94 432 OUT2 142

×

SimpleExtractChesterJaneBFCL/live_simple_189-114-0 (extractor.extract_information): [Chester,42],[Jane,43] (Jane a year older).1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 349 OUT183

×

SimpleServiceIdIroningUnitIntBFCL/live_simple_174-100-0 (get_service_id): ironing service -> service_id=2, unit omitted/1. Thai prompt.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN40 581 OUT643

×

SimpleWeatherNewYorkBFCL/live_simple_193-116-1 (Weather_1_GetWeather): city=New York, date=2023-03-08.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 856 OUT433

×

SimpleWeatherPacificaBFCL/live_simple_192-116-0 (Weather_1_GetWeather): city=Pacifica, date=2023-04-11.1 AI query 10 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN33 646 OUT883

Tool Calling - Simple: Live Services, Weather & Commands (set 10) 3 passed6 failed

33.3%

IN131 939 OUT3 838

×

SimpleCmdDockerPsBFCL/live_simple_172-99-6 (cmd_controller.execute): is Docker running -> 'docker ps'.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN18 312 OUT351

×

SimpleCmdDockerStartBFCL/live_simple_168-99-2 (cmd_controller.execute): start Docker -> 'docker start'.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 672 OUT170

✓

SimpleCmdEchoDateBFCL/live_simple_171-99-5 (cmd_controller.execute): today's date -> 'echo %date%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 586 OUT171

✓

SimpleCmdEchoTimeBFCL/live_simple_166-99-0 (cmd_controller.execute): current time -> 'echo %time%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 598 OUT238

×

SimpleCmdListCDriveBFCL/live_simple_170-99-4 (cmd_controller.execute): list C drive -> 'dir C:\'.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN9 613 OUT276

×

SimpleCmdRunPythonScriptBFCL/live_simple_169-99-3 (cmd_controller.execute): run python script with venv interpreter.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: (args.command?.ToLower()).AnyOf(["d:/playground/pc_contoller/env/scripts/python.exe d:/playground/pc_contoller/test.py"])

Failed

IN4 662 OUT153

✓

SimpleCmdStartCalcBFCL/live_simple_173-99-7 (cmd_controller.execute): open calculator -> 'start calc'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 579 OUT55

×

SimpleCmdTasklistCodeBFCL/live_simple_167-99-1 (cmd_controller.execute): is VS Code running -> tasklist filter.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN28 782 OUT696

×

SimpleProvidersChonburiDateRangeBFCL/live_simple_186-111-0 (get_service_providers): ironing, Chonburi, rating>=4.5, 12/03/2024 16:00-18:00.1 AI query 12 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN46 135 OUT1 728

Tool Calling - Simple: Live Services, Weather & Commands (set 11) 2 passed2 failed

50.0%

IN410 097 OUT7 837

✓

SimpleItemsCircleLogosBFCL/live_simple_162-96-1 (get_items): 4 circle-logo copyright items.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 830 OUT168

×

SimpleItemsLegoMovieBFCL/live_simple_163-96-2 (get_items): 7 Lego-movie copyright items.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN358 583 OUT5 739

✓

SimpleItemsSuperheroLogosBFCL/live_simple_161-96-0 (get_items): 12 superhero-logo copyright items.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 059 OUT268

×

SimpleProvidersChonburiNoLateBFCL/live_simple_187-112-0 (get_service_providers): ironing, Chonburi, rating>=4.5, 12/03/2024, never late. NOTE: the prompt gives a time range "16:00 - 18:00", so end_available_date is expected; the 27B occasionally omits it (~1/3 of runs) -> a legit model-reliability miss, not an assertion flaw.1 AI query 11 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN41 625 OUT1 662

Tool Calling - Simple: Live Services, Weather & Commands (set 12) 2 failed

0.0%

IN78 086 OUT1 940

×

SimpleCopyrightCocaColaBFCL/live_simple_164-97-0 (get_copyright_info): Coca-Cola logo, The Coca-Cola Company, confidence 100.0.1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN62 613 OUT1 548

×

SimpleProfessionalCleaningRating2BFCL/live_simple_183-108-0 (getDataForProfessional): cleaning in Bangkok, rating>=2.0 -> avg_rating=2.0, province_id=1, service_id=1.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN15 473 OUT392

Tool Calling - Simple: Live Services, Weather & Commands (set 13) 3 failed

0.0%

IN108 642 OUT2 813

×

SimpleMpnSpecs3pak7BFCL/live_simple_191-115-1 (raptor.mpn.specs): identifier=3pak7, MPN, include_images omitted/false.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN21 720 OUT689

×

SimpleMpnSpecsAchdImagesBFCL/live_simple_190-115-0 (raptor.mpn.specs): identifier=ACHD, MPN, include_images=true.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 496 OUT505

×

SimpleProfessionalCleaningRating4BFCL/live_simple_184-109-0 (getDataForProfessional): cleaning in Bangkok, rating>4 -> service_id=1, province_id=1, rating=4.0.1 AI query 19 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 19

Failed

IN74 426 OUT1 619

Tool Calling - Simple: Live Services, Weather & Commands (set 14) 1 passed1 failed

50.0%

IN21 538 OUT672

✓

SimpleExtractChineseNamesBFCL/live_simple_165-98-0 (extractor.extract_information): Chinese text -> [李雷,18],[李丽,21] (sister 3 yrs older).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN4 877 OUT118

×

SimpleProfessionalIroningChiangMaiBFCL/live_simple_188-113-0 (getDataForProfessional): ironing in Chiang Mai, Sankampang/Ton Pao, rating 4.5.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN16 661 OUT554

Tool Calling - Simple: Live Services, Weather & Commands (set 2) 3 failed

0.0%

IN57 270 OUT1 454

×

SimpleServiceIdIroningUnitSessionBFCL/live_simple_175-101-0 (get_service_id): ironing -> service_id=2, unit omitted/session. Thai prompt.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN20 713 OUT413

×

SimpleWeatherMartinezBFCL/live_simple_194-116-2 (Weather_1_GetWeather): city=Martinez, date=2023-04-25.1 AI query 8 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN24 551 OUT640

×

SimpleWeatherPaloAltoBFCL/live_simple_195-116-3 (Weather_1_GetWeather): city=Palo Alto, date=2023-04-25.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 006 OUT401

Tool Calling - Simple: Live Services, Weather & Commands (set 3) 3 failed

0.0%

IN135 302 OUT2 440

×

SimpleServiceIdIroningUnitQtyBFCL/live_simple_176-102-0 (get_service_id): ironing -> service_id=2, unit omitted/1. Thai prompt.1 AI query 25 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 25

Failed

IN114 021 OUT1 750

×

SimpleWeatherAlamedaWednesdayBFCL/live_simple_196-116-4 (Weather_1_GetWeather): city=Alameda, upcoming Wednesday (today Mon 2023-04-25) -> 2023-04-27.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN9 290 OUT334

×

SimpleWeatherStinsonBeachBFCL/live_simple_197-116-5 (Weather_1_GetWeather): city=Stinson Beach, date=2023-04-05.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN11 991 OUT356

Tool Calling - Simple: Live Services, Weather & Commands (set 4) 4 failed

0.0%

IN83 462 OUT1 776

×

SimpleServiceIdCleaningBFCL/live_simple_177-103-0 (get_service_id): cleaning -> service_id=1, unit omitted/1.1 AI query 12 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN37 968 OUT692

×

SimpleServiceIdIroningBFCL/live_simple_178-103-1 (get_service_id): ironing -> service_id=2, unit omitted/1.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN15 715 OUT284

×

SimpleWeatherHealdsburgBFCL/live_simple_198-116-6 (Weather_1_GetWeather): city=Healdsburg, date=2023-03-02.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN14 912 OUT403

×

SimpleWeatherMarshallBFCL/live_simple_199-116-7 (Weather_1_GetWeather): city=Marshall (prompt 'Marshall, MN'), date=2023-03-05. NOTE: GT city is bare "Marshall"; model faithfully copies the "Marshall, MN" disambiguator from the prompt -> substring-intent (equivalent-correct).1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN14 867 OUT397

Tool Calling - Simple: Live Services, Weather & Commands (set 5) 3 failed

0.0%

IN46 096 OUT1 159

×

SimpleServiceIdIroningChiangMaiBFCL/live_simple_179-104-0 (get_service_id): ironing in Chiang Mai -> service_id=2, province_id=2.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN9 256 OUT189

×

SimpleWeatherCampbellBFCL/live_simple_201-116-9 (Weather_1_GetWeather): city=Campbell, date=2023-03-04.1 AI query 8 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN24 749 OUT641

×

SimpleWeatherFremontBFCL/live_simple_200-116-8 (Weather_1_GetWeather): city=Fremont, date=2023-03-01.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 091 OUT329

Tool Calling - Simple: Live Services, Weather & Commands (set 6) 3 failed

0.0%

IN110 521 OUT2 350

×

SimpleServiceIdIroningBangkokRatingBFCL/live_simple_180-105-0 (get_service_id): ironing in Bangkok, rating>=4 -> service_id=2, province_id=1, rating=4.0.1 AI query 19 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 19

Failed

IN73 306 OUT1 383

×

SimpleWeatherFosterCityBFCL/live_simple_202-116-10 (Weather_1_GetWeather): city=Foster City, date=2023-04-25.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN12 241 OUT359

×

SimpleWeatherWashingtonBFCL/live_simple_203-116-11 (Weather_1_GetWeather): city=Washington (prompt 'Washington, DC'), date=2023-03-01. NOTE: GT city is bare "Washington"; model faithfully copies the "Washington, DC" disambiguator from the prompt -> substring-intent (equivalent-correct).1 AI query 8 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN24 974 OUT608

Tool Calling - Simple: Live Services, Weather & Commands (set 7) 3 failed

0.0%

IN94 207 OUT2 242

×

SimpleServiceIdIroningChatuchakBFCL/live_simple_181-106-0 (get_service_id): ironing, Bangkok(Chatuchak), rating>=4.5 -> service_id=2, province_id omitted/1, district_name=Chatuchak, rating=4.5.1 AI query 17 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 17

Failed

IN65 645 OUT1 407

×

SimpleWeatherBerkeleySaturdayBFCL/live_simple_205-116-13 (Weather_1_GetWeather): city=Berkeley, upcoming Saturday (today Tue 2023-04-25) -> 2023-04-29.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN15 464 OUT550

×

SimpleWeatherRutherfordBFCL/live_simple_204-116-12 (Weather_1_GetWeather): city=Rutherford (prompt 'Rutherford, NJ'), date=2023-04-22. NOTE: GT city is bare "Rutherford"; model faithfully copies the "Rutherford, NJ" disambiguator from the prompt -> substring-intent (equivalent-correct).1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 098 OUT285

Tool Calling - Simple: Live Services, Weather & Commands (set 8) 3 failed

0.0%

IN56 996 OUT1 595

×

SimpleProvidersKhlongSamWaBFCL/live_simple_182-107-0 (get_service_providers): ironing, Bangkok, Khlong Sam Wa / Sai Kong, rating>=4.5.1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN34 372 OUT980

×

SimpleWeatherLondonBFCL/live_simple_206-116-14 (Weather_1_GetWeather): city=London (prompt 'London, England'), date=2023-03-05.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 082 OUT189

×

SimpleWeatherSacramentoBFCL/live_simple_207-116-15 (Weather_1_GetWeather): city=Sacramento, date=2023-04-22.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN15 542 OUT426

Tool Calling - Simple: Live Services, Weather & Commands (set 9) 4 passed2 failed

66.7%

IN85 959 OUT2 059

✓

SimpleCmdEchoTimeUnitBFCL/live_simple_157-95-14 (cmd_controller.execute): current time -> 'echo %time%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 623 OUT180

×

SimpleCmdListCDriveUnitBFCL/live_simple_156-95-13 (cmd_controller.execute): list C drive -> 'dir C:\'.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN19 371 OUT289

✓

SimpleCmdStartCameraBFCL/live_simple_160-95-17 (cmd_controller.execute): open camera -> 'start microsoft.windows.camera:'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 618 OUT57

✓

SimpleCmdTaskkillCodeBFCL/live_simple_158-95-15 (cmd_controller.execute): kill VS Code -> 'taskkill /IM code.exe /F'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 618 OUT60

✓

SimpleCmdTimeout10BFCL/live_simple_159-95-16 (cmd_controller.execute): timer 10s -> 'timeout 10'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 638 OUT87

×

SimpleProvidersChonburiTomorrowBFCL/live_simple_185-110-0 (get_service_providers): ironing, Chonburi, rating>=4.5, tomorrow of 20230929 -> 2023-09-30.1 AI query 13 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 13

Failed

IN48 091 OUT1 386

Tool Calling - Simple: Live Shell, Classification & APIs (set 1) 1 passed2 failed

33.3%

IN42 701 OUT1 295

✓

SimpleClassifyGreetingsAndLinksBFCL/live_simple_105-62-0 (classify): clean_hello=['hello'], faq_link_accounts_start=['link my account','connect my accounts'].1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 601 OUT94

×

SimpleCmdDockerVersionBFCL/live_simple_141-94-0 (cmd_controller.execute): command='docker --version'.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN22 137 OUT409

×

SimpleReqNodesPekAciBFCL/live_simple_128-83-0 (requests.get): nodes list url with ip 192.168.11.33, fabricName='PEK-ACI'.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN15 963 OUT792

Tool Calling - Simple: Live Shell, Classification & APIs (set 10) 2 passed2 failed

50.0%

IN26 350 OUT1 561

✓

SimpleCmdShutdownBFCL/live_simple_150-95-7 (cmd_controller.execute): shutdown the pc via 'shutdown /s /t 0'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 870 OUT77

×

SimpleProcessImageMoondreamBFCL/live_simple_122-78-0 (process_data): image.png + long question + model 'vikhyatk/moondream2'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 685 OUT402

×

SimpleReqDataLimit50BFCL/live_simple_135-88-0 (requests.get): data endpoint url, params limit=50.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN8 927 OUT1 013

✓

SimpleSumNumbersListBFCL/live_simple_116-72-0 (sum_numbers): numbers_list=[133, 34].1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 868 OUT69

Tool Calling - Simple: Live Shell, Classification & APIs (set 11) 1 passed3 failed

25.0%

IN47 010 OUT1 703

✓

SimpleCmdWifiDisconnectBFCL/live_simple_151-95-8 (cmd_controller.execute): disconnect wifi via 'netsh wlan disconnect'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 897 OUT57

×

SimplePipelineFurnitureImageBFCL/live_simple_123-79-0 (pipeline): analyze thewowstyle image with furniture/minimalist question.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN11 181 OUT779

×

SimpleReqTopologySalesAppBFCL/live_simple_136-89-0 (requests.get): topology url ip 192.168.1.1, filter accountName:AcmeCorp AND applicationName:SalesApp.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN20 598 OUT558

×

SimpleTemperatureHyderabadBFCL/live_simple_118-74-0 (get_temperature): units=Celsius, location='Hyderabad, India', time omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 334 OUT309

Tool Calling - Simple: Live Shell, Classification & APIs (set 12) 1 passed3 failed

25.0%

IN63 296 OUT1 946

×

SimpleCmdDirDAndEchoBFCL/live_simple_152-95-9 (cmd_controller.execute): 'dir D:\ && echo testing.txt'.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN31 949 OUT683

×

SimpleReqAnomaliesSummaryBFCL/live_simple_139-92-0 (requests.get): anomalies summary url ip 192.168.1.1, params fabric='network222'.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN11 426 OUT511

✓

SimpleRescheduleAlice8pmBFCL/live_simple_137-90-0 (reschedule): identifier='Alice-One-one-One', 2023-11-01T20:00:00, Europe/London.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 310 OUT197

×

SimpleWeatherForecastParisBFCL/live_simple_120-76-0 (weather.forecast): Paris, France 2023-04-03..05, Celsius, defaults omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN14 611 OUT555

Tool Calling - Simple: Live Shell, Classification & APIs (set 13) 2 passed1 failed

66.7%

IN22 951 OUT602

×

SimpleCmdDelPuttyBFCL/live_simple_153-95-10 (cmd_controller.execute): remove putty shortcut via del with quoted path.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 051 OUT237

✓

SimpleDynamicPricingSaaSBFCL/live_simple_113-69-0 (calculate_dynamic_pricing): geolocation='34.0522, -118.2437', base=100, min=90.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 042 OUT246

✓

SimpleRescheduleBob4pmBFCL/live_simple_138-91-0 (reschedule): identifier='Bob-123', 2023-11-01T16:00:00, Europe/London.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 858 OUT119

Tool Calling - Simple: Live Shell, Classification & APIs (set 14) 3 failed

0.0%

IN69 974 OUT1 736

×

SimpleCmdStartYoutubeBFCL/live_simple_154-95-11 (cmd_controller.execute): open youtube rickroll via start command.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 261 OUT165

×

SimpleGetTicketsCompanyOpenBFCL/live_simple_124-80-0 (get_tickets): customer (company name in prompt), status=open, limit=10.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN33 202 OUT823

×

SimpleGreetPartnerJeffBFCL/live_simple_140-93-0 (greet_partner): user='Emily', partner='Jeff', relationship='husband'.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN29 511 OUT748

Tool Calling - Simple: Live Shell, Classification & APIs (set 15) 3 failed

0.0%

IN19 658 OUT1 370

×

SimpleCalculateTaxNewYorkBFCL/live_simple_104-61-2 (calculate_tax): purchase_amount=250, state='NY', all optionals omitted/default.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 427 OUT291

×

SimpleCmdDateBFCL/live_simple_155-95-12 (cmd_controller.execute): what is the date via 'date /t' or 'echo %date%'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: args.command.ToLower().AnyOf(["date /t", "echo %date%"])

Failed

IN4 807 OUT960

×

SimpleConcatJohnDoeBFCL/live_simple_125-81-0 (concat_strings): strings=['John','Doe'], separator=' '. EXPECTED MODEL MISS: 'combine ... into a full name' requires separator=' ' (John Doe). The 27B tends to omit separator (=> 'JohnDoe'); that is a legit model failure of the measured competence, so the ' ' assertion stays strict.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 424 OUT119

Tool Calling - Simple: Live Shell, Classification & APIs (set 2) 3 failed

0.0%

IN37 341 OUT2 244

×

SimpleCmdDirDesktopBFCL/live_simple_142-94-1 (cmd_controller.execute): command='dir Desktop'.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 572 OUT326

×

SimpleRecordBankNineIntentsBFCL/live_simple_106-63-0 (record): classify 20 bank queries into 9 intents (2 buckets empty).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: args.activate_card_start.Count == 2 Values: args.activate_card_start.Count = 1

Failed

IN5 955 OUT1 032

×

SimpleReqNodesFabEdBFCL/live_simple_129-83-1 (requests.get): nodes list url with ip 3.321.3232.2, fabricName='fab-ed'.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN20 814 OUT886

Tool Calling - Simple: Live Shell, Classification & APIs (set 3) 2 passed2 failed

50.0%

IN47 337 OUT1 710

✓

SimpleCmdDockerPsBFCL/live_simple_143-95-0 (cmd_controller.execute): command='docker ps'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 221 OUT319

✓

SimpleRecordTelephoneHoursAccountsBFCL/live_simple_107-64-0 (record): classify 7 queries into telephone-banking / bank-hours / accounts.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 644 OUT339

×

SimpleTelemetryGlobalFabricStatusBFCL/live_simple_126-82-0 (telemetry.flowrules.interfaceInfo.get): fabric='Global-Fabric', node=1200, pod=3, ethernet, status.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN11 254 OUT373

×

SimpleUpdateProfileEmailAgeBFCL/live_simple_114-70-0 (update_user_profile): user_id=12345, email+age set, bio omitted, notify default.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN25 218 OUT679

Tool Calling - Simple: Live Shell, Classification & APIs (set 4) 1 passed3 failed

25.0%

IN48 921 OUT1 496

✓

SimpleCmdKillFirefoxBFCL/live_simple_144-95-1 (cmd_controller.execute): close firefox via taskkill.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 085 OUT66

×

SimpleRecordMoneyTransactionsCreditBFCL/live_simple_108-65-0 (record): classify 10 queries into money-movement / transactions / credit-limit / link-accounts. EXPECTED MODEL MISS: the 27B classifies 'show me all outgoing wire transfers' as a transaction-view (get_transactions_start) rather than money_movement_start per GT, so the get_transactions count comes out as 5 instead of 4. GT bucketing kept strict; this is a legit model classification failure.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN15 873 OUT869

×

SimpleReverseSayHiBFCL/live_simple_117-73-0 (reverse_input): input_value='say hi'.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN19 996 OUT287

×

SimpleTelemetryTestDeStatusBFCL/live_simple_127-82-1 (telemetry.flowrules.interfaceInfo.get): fabric='test-de', node=5, pod=1, ethernet, status.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 967 OUT274

Tool Calling - Simple: Live Shell, Classification & APIs (set 5) 3 failed1 broken

0.0%

IN337 708 OUT19 097

!

SimpleCmdDirCDriveBFCL/live_simple_145-95-2 (cmd_controller.execute): list C drive => 'dir C:\'. EXPECTED MODEL MISS: for the terse/typo'd 'LIST C DRIIVE' the 27B currently returns 'dir d:' — the wrong drive. The accepted-command list is broadened with the no-backslash 'dir c:' variant for fairness, but a wrong drive letter is a legit model failure.1 AI query 28 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Broken

IN306 863 OUT17 721

×

SimplePrimeMinistersThreeBFCL/live_simple_119-75-0 (GetPrimeMinisters): countries=[Australia, Canada, India].1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 885 OUT208

×

SimpleRecordAutoWithdrawIncomeBFCL/live_simple_109-66-0 (record): classify 5 loan/withdraw queries; outofscope empty.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN11 594 OUT538

×

SimpleReqInterfaceStatsNetwork1BFCL/live_simple_130-84-0 (requests.get): interfaceInfo stats, ip 192.120.45.67, Network1/node 12/pod 10/ethernet/statistics.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN11 366 OUT630

Tool Calling - Simple: Live Shell, Classification & APIs (set 6) 1 passed3 failed

25.0%

IN36 630 OUT1 093

×

SimpleCmdStartHuggingfaceBFCL/live_simple_146-95-3 (cmd_controller.execute): open huggingface via start command.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN8 092 OUT118

×

SimpleGenerateSalaryChartBFCL/live_simple_121-77-0 (generate_chart): 5 job labels and salaries in order.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN8 349 OUT228

✓

SimpleRecordAtmLocationOnlyBFCL/live_simple_110-67-0 (record): single query 'Need ATM location' -> atm_finder_start only.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 295 OUT102

×

SimpleReqInterfaceSpeedNetwork1BFCL/live_simple_131-84-1 (requests.get): interfaceInfo Speed, ip 192.120.45.67, Network1/node 12/pod 10/ethernet/Speed.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN14 894 OUT645

Tool Calling - Simple: Live Shell, Classification & APIs (set 7) 2 passed1 failed

66.7%

IN24 441 OUT920

✓

SimpleCmdKillTimerBFCL/live_simple_147-95-4 (cmd_controller.execute): remove timer.exe via taskkill.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 869 OUT71

✓

SimpleRecordBalanceAndAtmBFCL/live_simple_111-67-1 (record): 10 queries -> 8 balance + 2 atm; other buckets empty.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN5 204 OUT372

×

SimpleReqDeviceConnectorVersionsBFCL/live_simple_132-85-0 (requests.get): device connector versions JSON url, no params.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN14 368 OUT477

Tool Calling - Simple: Live Shell, Classification & APIs (set 8) 1 passed2 failed

33.3%

IN334 940 OUT4 439

×

SimpleCmdWhatFileInCBFCL/live_simple_148-95-5 (cmd_controller.execute): 'what is the file in c' => 'dir C:\'. EXPECTED MODEL MISS: the 27B mis-selects the 'record' classifier (dropping the request into its 'outofscope' bucket) instead of cmd_controller_execute. record is a legitimate distractor here (not a looping universal collider), so this stays as a real tool-selection failure — no isolation needed.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN322 299 OUT4 089

✓

SimpleRecordWeatherOutOfScopeBFCL/live_simple_112-68-0 (record): single query 'what is the weather like' -> outofscope only.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN4 962 OUT135

×

SimpleReqRecommendationsDacBFCL/live_simple_133-86-0 (requests.get): recommendations url, params advisoryId='dac'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 679 OUT215

Tool Calling - Simple: Live Shell, Classification & APIs (set 9) 2 passed1 failed

66.7%

IN16 432 OUT456

✓

SimpleCalculateSum133And34BFCL/live_simple_115-71-0 (calculate_sum): number1=133, number2=34.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 611 OUT74

✓

SimpleCmdEchoHiBFCL/live_simple_149-95-6 (cmd_controller.execute): say hi via echo command.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN4 585 OUT62

×

SimpleReqInsightsGroupsBFCL/live_simple_134-87-0 (requests.get): insights groups url, params limit=20, sort='desc'.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN7 236 OUT320

Tool Calling - Simple: Math, Physics & Geometry (set 1) 12 failed

0.0%

IN422 084 OUT7 304

×

SimpleAlgebraQuadraticRootsBFCL/simple_python_3 (algebra.quadratic_roots): a=1, b=-3, c=2.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 273 OUT199

×

SimpleAreaUnderCurveXSquaredBFCL/simple_python_13 (calculate_area_under_curve): function y=x^2, interval [1,3], method omitted/trapezoidal.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN53 187 OUT867

×

SimpleCircleCircumferenceInchesBFCL/simple_python_7 (calculate_circumference): radius=4, unit inches/in.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 236 OUT304

×

SimpleDerivativeCubicPolyBFCL/simple_python_14 (calculate_derivative): function 3x^2+2x-1, x_value omitted or 0.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 687 OUT408

×

SimpleElectricFieldFromChargeBFCL/simple_python_39 (calculate_electric_field): charge=2, distance=3, permitivity omitted/8.854e-12.1 AI query 11 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN52 514 OUT928

×

SimpleFinalVelocityFromDistanceBFCL/simple_python_30 (kinematics.final_velocity_from_distance): acceleration=4, distance=300, initial_velocity omitted/0.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 550 OUT395

×

SimpleFinalVelocityFromHeightBFCL/simple_python_25 (calculate_final_velocity): height=150, initial_velocity omitted/0, gravity omitted/9.81.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN28 325 OUT574

×

SimpleGeometryAreaCircleBFCL/simple_python_8 (geometry.area_circle): radius=10, units omitted or 'meters'.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 201 OUT164

×

SimpleGetPrimeFactors450BFCL/simple_python_17 (get_prime_factors): number=450, formatted omitted or true.1 AI query 19 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 19

Failed

IN98 796 OUT1 265

×

SimpleMagneticFieldLoopBFCL/simple_python_40 (calculate_magnetic_field): current=5, radius=4, permeability omitted/12.57e10.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN28 165 OUT1 114

×

SimpleMathGcd40And50BFCL/simple_python_19 (math.gcd): num1=40, num2=50.1 AI query 10 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN47 462 OUT606

×

SimpleTriangleAreaWithUnitBFCL/simple_python_0 (calculate_triangle_area): base=10, height=5, unit omitted or 'units'.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN32 688 OUT480

Tool Calling - Simple: Math, Physics & Geometry (set 2) 1 passed11 failed

8.3%

IN375 409 OUT7 602

×

SimpleAverageVelocityBFCL/simple_python_26 (calculate_velocity): distance=50, duration=2, unit omitted or 'km/h'.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN28 018 OUT420

×

SimpleCalculusDerivativeAtValueBFCL/simple_python_16 (calculus.derivative): function 2x^2, value=1, function_variable omitted or 'x'.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 606 OUT388

×

SimpleElectricFieldStrengthBFCL/simple_python_44 (calculate_electric_field_strength): charge=0.01, distance=4, medium omitted or 'vacuum'.1 AI query 10 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN47 407 OUT860

×

SimpleFinalVelocityConstAccelBFCL/simple_python_31 (calculate_final_velocity): initial_velocity=0, acceleration=9.8, time=5.1 AI query 10 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN47 955 OUT746

×

SimpleGeometryCalculateAreaCircleBFCL/simple_python_9 (geometry.calculate_area_circle): radius=5, unit omitted or 'units'.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN20 496 OUT270

✓

SimpleGeometryCircumferenceBFCL/simple_python_12 (geometry.circumference): radius=3, units omitted or 'cm'.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 331 OUT115

×

SimpleIntegrateCubeSimpsonBFCL/simple_python_15 (integrate): function x^3, start_x=-2, end_x=3, method simpson.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 561 OUT1 073

×

SimpleMagneticFieldStrengthWireBFCL/simple_python_43 (calculate_magnetic_field_strength): current=20, distance=10, permeability omitted/12.57e-7.1 AI query 8 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN37 457 OUT928

×

SimpleMathHcfBFCL/simple_python_20 (math.hcf): number1=36, number2=24.1 AI query 10 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN47 275 OUT664

×

SimpleNumberAnalysisPrimeFactorsBFCL/simple_python_18 (number_analysis.prime_factors): number=123456.1 AI query 12 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN57 341 OUT1 345

×

SimpleRightTriangleAreaCmBFCL/simple_python_10 (calculate_area): base=6, height=10, unit omitted or 'cm'.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 260 OUT239

×

SimpleSolveQuadraticEquation265BFCL/simple_python_4 (solve_quadratic_equation): a=2, b=6, c=5.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 702 OUT554

Tool Calling - Simple: Math, Physics & Geometry (set 3) 9 failed

0.0%

IN240 669 OUT5 356

×

SimpleBoilingMeltingPointsBFCL/simple_python_47 (get_boiling_melting_points): substance='water', sea_level=5000.1 AI query 22 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 22

Failed

IN113 067 OUT1 677

×

SimpleDisplacementBFCL/simple_python_28 (calculate_displacement): initial_velocity=10, time=5, acceleration=9.8.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 928 OUT369

×

SimpleElectrostaticPotentialBFCL/simple_python_38 (calculate_electrostatic_potential): charge1=1e-9, charge2=2e-9, distance=0.05, constant omitted/8.99e9.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 280 OUT1 062

×

SimpleFinalSpeedFromHeightBFCL/simple_python_32 (calculate_final_speed): initial_velocity=0, height=100, gravity omitted or 9.8.1 AI query 7 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN29 908 OUT517

×

SimpleNumberTheoryGcdBFCL/simple_python_21 (number_theory.gcd): number1=36, number2=48.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 866 OUT191

×

SimplePrimeFactorizeDictionaryBFCL/simple_python_23 (prime_factorize): number=60, return_type='dictionary'.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 636 OUT284

×

SimpleRouteEstimateTimeStopsBFCL/simple_python_37 (route.estimate_time): SF -> LA, stops {Santa Barbara, Monterey} in either order.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 849 OUT431

×

SimpleSolveQuadraticAllRootsBFCL/simple_python_5 (solve_quadratic): a=3, b=-11, c=-4, root_type='all' (user asked for all roots).1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 954 OUT614

×

SimpleTriangleAreaNoUnitBFCL/simple_python_11 (calculate_triangle_area): base=10, height=5.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 181 OUT211

Tool Calling - Simple: Math, Physics & Geometry (set 4) 2 passed7 failed

22.2%

IN379 042 OUT7 251

×

SimpleCalculateDensityBFCL/simple_python_48 (calculate_density): mass=45, volume=15, unit omitted or 'kg/m^3'.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 127 OUT189

×

SimpleElectromagneticForceBFCL/simple_python_41 (electromagnetic_force): charge1=5, charge2=7, distance=3, medium_permittivity omitted/8.854e-12.1 AI query 10 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN43 755 OUT904

✓

SimpleFinalSpeedFreeFallTimeBFCL/simple_python_29 (calculate_final_speed): initial_speed omitted/0, time=5, gravity omitted/-9.81.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 631 OUT349

✓

SimpleMathFactorial5BFCL/simple_python_1 (math.factorial): number=5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 604 OUT56

×

SimpleMathGcd12And15BFCL/simple_python_22 (math.gcd): num1=12, num2=15.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 121 OUT129

×

SimpleShortestDrivingDistanceBFCL/simple_python_36 (get_shortest_driving_distance): origin=New York City, destination=Washington D.C., unit omitted/km.1 AI query 12 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN53 501 OUT923

×

SimpleSolveQuadraticRoots253BFCL/simple_python_6 (solve_quadratic): a=2, b=5, c=3.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 571 OUT416

×

SimpleThermoEnergyVaporizationBFCL/simple_python_45 (thermo.calculate_energy): mass=100, phase_transition=vaporization, substance omitted/water.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 726 OUT365

×

SimpleTravelItineraryTokyoBFCL/simple_python_34 (travel_itinerary_generator): destination=Tokyo, days=7, daily_budget=100, exploration_type=nature.1 AI query 34 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 34

Failed

IN213 006 OUT3 920

Tool Calling - Simple: Math, Physics & Geometry (set 5) 8 failed

0.0%

IN254 104 OUT5 176

×

SimpleAbsolutePressureBFCL/simple_python_49 (calc_absolute_pressure): atm_pressure=1, gauge_pressure=2.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 855 OUT366

×

SimpleFinalTemperatureMixBFCL/simple_python_46 (calculate_final_temperature): mass1=20, temperature1=30, mass2=15, temperature2=60, shc omitted/4.2.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN25 539 OUT849

×

SimpleFinalVelocityAccelTimeBFCL/simple_python_27 (final_velocity): initial_velocity=10, acceleration=2, time=5.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 884 OUT280

×

SimpleGetDirectionsFastestBFCL/simple_python_33 (get_directions): start=Sydney, end=Melbourne, route_type omitted or 'fastest'.1 AI query 9 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN38 066 OUT790

×

SimpleMathGcd12And18BFCL/simple_python_24 (math.gcd): num1=12, num2=18.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN20 837 OUT296

×

SimpleMathHypotenuseBFCL/simple_python_2 (math.hypot): x=4, y=5, z omitted or 0.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 338 OUT243

×

SimpleResonantFrequencyLCBFCL/simple_python_42 (calculate_resonant_frequency): inductance=0.05 (50mH), capacitance=0.0001 (100uF), round_off omitted/2.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 186 OUT638

×

SimpleVeganRestaurantFindBFCL/simple_python_35 (vegan_restaurant.find_nearby): location='New York, NY', operating_hours=23 (11 PM).1 AI query 22 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 22

Failed

IN110 399 OUT1 714

Tool Calling - Simple: Science, Data & Biology (set 1) 13 failed

0.0%

IN653 732 OUT10 752

×

SimpleArraySortBFCL/simple_python_87 (array_sort): list=[5,3,4,1,2], order=ascending.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN26 303 OUT344

×

SimpleAverageTemperatureBFCL/simple_python_78 (average_temperature): location=Austin, days=3, temp_unit=Celsius.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN12 476 OUT181

×

SimpleBacteriaEvolutionRateBFCL/simple_python_74 (calculate_bacteria_evolution_rate): start=5000, dup_freq=1, duration=6, gen_time omitted or 20.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN21 626 OUT667

×

SimpleBmiKgCmBFCL/simple_python_84 (calculate_bmi): weight=85, height=180, unit omitted or 'metric'.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN16 970 OUT346

×

SimpleCreateHistogramBFCL/simple_python_79 (create_histogram): data=[85,90,88,92,86,89,91], bins=5.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN26 627 OUT451

×

SimpleCreatureFitnessBFCL/simple_python_72 (calculate_fitness): trait_values=[0.8,0.7], trait_contributions=[0.4,0.6].1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN36 660 OUT572

×

SimpleDbFetchStudentsBFCL/simple_python_89 (db_fetch_records): database=StudentDB, table=students, conditions={department:Science, school:Bluebird High School}, fetch_limit omitted or 0.1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN53 584 OUT1 084

×

SimpleEntropyChangeSubstanceBFCL/simple_python_50 (entropy_change.calculate): substance=ice, mass=1, init=0, final=100, pressure omitted or 1.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN36 937 OUT1 248

×

SimpleGenerateDnaSequenceBFCL/simple_python_71 (generate_DNA_sequence): length=100, preferences=[G,C] in either order.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN26 252 OUT399

×

SimpleGpsCoordinateDistanceBFCL/simple_python_83 (calculate_distance): coord1=[33.4484,-112.074], coord2=[34.0522,-118.2437], unit=miles.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN12 686 OUT249

×

SimpleImdbMoviesByActorBFCL/simple_python_92 (imdb.find_movies_by_actor): actor='Leonardo DiCaprio', year=2010, category omitted or 'all'.1 AI query 29 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 29

Failed

IN191 805 OUT2 531

×

SimplePopulationProjectionBFCL/simple_python_73 (population_projections): country=United States/USA, years=20, growth_rate omitted or 1.2.1 AI query 14 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 14

Failed

IN75 086 OUT976

×

SimpleRestaurantFindNearbyBFCL/simple_python_77 (restaurant.find_nearby): location='Los Angeles, CA', dietary_preference=[Vegan].1 AI query 20 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 20

Failed

IN116 720 OUT1 704

Tool Calling - Simple: Science, Data & Biology (set 2) 3 passed10 failed

23.1%

IN957 593 OUT14 031

×

SimpleBmiMetersBFCL/simple_python_88 (calculate_BMI): weight_kg=70, height_m=1.75.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN16 904 OUT333

×

SimpleClockAngleBFCL/simple_python_98 (calculate_clock_angle): hours=6, minutes=30, round_to omitted or 2.1 AI query 34 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 34

Failed

IN226 376 OUT2 727

×

SimpleDatabaseQueryConditionsBFCL/simple_python_96 (database.query): table=user, conditions=[{age,>,25},{job,=,engineer}].1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN17 082 OUT387

×

SimpleElephantPopulationBFCL/simple_python_75 (elephant_population_estimate): current=35000, growth_rate=0.015, years=5.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN31 385 OUT484

×

SimpleEntropyChangeIsothermalBFCL/simple_python_51 (calculate_entropy_change): initial_temp=300, final_temp=400, heat_capacity=5, isothermal omitted or true.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN47 126 OUT806

×

SimpleEvolutionPredictionDarwinBFCL/simple_python_76 (prediction.evolution): species=Homo Sapiens, years=50, model=Darwin.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN16 932 OUT351

×

SimpleFindRestaurantsThaiBFCL/simple_python_80 (find_restaurants): location=Manhattan variants, food_type=Thai, number=5, dietary_requirements=[vegan].1 AI query 50 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN507 433 OUT7 160

✓

SimpleGeoDistanceCitiesBFCL/simple_python_85 (geo_distance.calculate): start='Boston, MA', end='Washington, D.C.', units omitted or 'miles'.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN8 201 OUT142

×

SimpleMathFactorialBFCL/simple_python_97 (math.factorial): number=5.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN16 738 OUT193

×

SimplePlotSineWaveBFCL/simple_python_99 (plot_sine_wave): start=0.0, end=6.2832, frequency=5, amplitude omitted or 1, phase_shift omitted or 0.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN26 327 OUT460

×

SimpleTheaterMovieReleasesBFCL/simple_python_93 (get_theater_movie_releases): location=LA, timeframe=7, format=IMAX.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN26 660 OUT699

✓

SimpleTriangleAreaMetersBFCL/simple_python_95 (calc_area_triangle): base=5, height=3.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN8 179 OUT164

✓

SimpleUpdateUserInfoBFCL/simple_python_94 (update_user_info): user_id=43523, update_info={name:John Doe, email:johndoe@email.com}, database omitted or CustomerInfo. NOTE: single-call contract kept intentionally; the model very occasionally emits a spurious second call (args are correct ~3/3 sampled runs), a model-reliability signal rather than a test defect.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN8 250 OUT125

Tool Calling - Simple: Science, Data & Biology (set 3) 1 passed11 failed

8.3%

IN715 283 OUT12 045

×

SimpleCalculateAverageBFCL/simple_python_82 (calculate_average): numbers=[12,15,18,20,21,26,30].1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 704 OUT466

×

SimpleCellDensityBFCL/simple_python_57 (calculate_cell_density): optical_density=0.6, dilution=5, calibration_factor omitted or 1e9.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 560 OUT874

×

SimpleCellInfoHumanBFCL/simple_python_55 (biology.get_cell_info): cell_type=human, detailed=true.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN378 104 OUT4 031

×

SimpleCellProteinsMembraneBFCL/simple_python_56 (cellbio.get_proteins): cell_compartment=plasma membrane, include_description omitted/true/false.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 958 OUT783

×

SimpleCityDistanceTrainBFCL/simple_python_86 (city_distance.find_shortest): start=New York, end=Los Angeles, transportation=train, allow_transfer=true.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN30 518 OUT521

×

SimpleEmployeeFetchDataBFCL/simple_python_90 (employee.fetch_data): company='ABC Ltd.', employee_id=345, data_field=[Personal Info, Job History].1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN20 224 OUT566

✓

SimpleFetchDnaSequenceBFCL/simple_python_53 (fetch_DNA_sequence): DNA_id=DNA123, format omitted or 'fasta', upstream omitted or 0.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 603 OUT225

×

SimpleGetRestaurantSushiBFCL/simple_python_91 (get_restaurant): cuisine=sushi, location=Boston, condition=open on Sundays.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 510 OUT339

×

SimpleHeatCapacityAirBFCL/simple_python_52 (calc_heat_capacity): temp=298, volume=10, gas omitted or 'air'.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN29 153 OUT1 320

×

SimpleMolecularWeightBFCL/simple_python_59 (calculate_molecular_weight): compound=C6H12O6, to_unit=grams/mole or g/mol.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN20 055 OUT402

×

SimpleOrganelleFunctionLookupBFCL/simple_python_58 (cell_biology.function_lookup): molecule=ATP synthase, organelle=mitochondria, specific_function=true.1 AI query 20 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 20

Failed

IN111 548 OUT1 840

×

SimpleProteinSequenceBFCL/simple_python_54 (get_protein_sequence): gene=BRCA1, species omitted or 'Homo sapiens'.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 346 OUT678

Tool Calling - Simple: Science, Data & Biology (set 4) 12 failed

0.0%

IN669 193 OUT12 753

×

SimpleAnalyzeDnaSequenceBFCL/simple_python_62 (analyze_dna_sequence): sequence + reference_sequence given, mutation_type omitted or 'substitution'.1 AI query 15 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN82 398 OUT1 796

×

SimpleDiabetesPredictionBFCL/simple_python_61 (diabetes_prediction): weight=150, height=70 (5ft10in), activity_level=lightly active.1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN30 020 OUT1 367

×

SimpleFastestRouteAvoidTollsBFCL/simple_python_81 (map_routing.fastest_route): start=San Francisco/SF, end=Los Angeles/LA, avoid_tolls=true.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN11 933 OUT201

×

SimpleForestGrowthForecastBFCL/simple_python_68 (forest_growth_forecast): location=Yellowstone National Park, years=5, include_human_impact=true.1 AI query 20 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 20

Failed

IN113 081 OUT1 684

×

SimpleGeneticSimilarityBFCL/simple_python_63 (genetics.calculate_similarity): species1=Human, species2=Chimp/Chimpanzee, format omitted or 'percentage'.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN31 102 OUT478

×

SimpleGenotypeFrequencyBFCL/simple_python_64 (calculate_genotype_frequency): allele_frequency=0.3, genotype=AA.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN25 109 OUT483

×

SimpleIdentifyBirdBFCL/simple_python_67 (identify_bird): color=green, habitat=forest, size=small.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN34 640 OUT1 028

×

SimpleMutationTypeFindBFCL/simple_python_60 (mutation_type.find): snp_id=rs6034464, species omitted or 'Homo sapiens'.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN25 073 OUT456

×

SimplePopulationDensityBFCL/simple_python_65 (calculate_density): country=Brazil, year='2022', population=213000000, land_area=8500000.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN16 376 OUT315

×

SimplePrecipitationStatsBFCL/simple_python_66 (ecology_data.precipitation_stats): location=Amazon rainforest, time_frame=six_months.1 AI query 31 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 31

Failed

IN197 998 OUT2 954

×

SimpleTurtlePopulationBFCL/simple_python_69 (ecology.get_turtle_population): location=Mississippi river, year=2020, species=true.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN61 774 OUT1 277

×

SimpleVehicleEmissionBFCL/simple_python_70 (calculate_vehicle_emission): vehicle_type=gas, miles_driven=1500, emission_factor omitted or 355.48.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN39 689 OUT714

Tool Calling - Simple: Sports & Games (set 1) 2 passed10 failed

16.7%

IN277 340 OUT7 203

✓

SimpleBattingAverageComputeBFCL/simple_python_302 (calculate_batting_average): hits=180, at_bats=600, decimal_places=3 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 671 OUT182

×

SimpleBoardGameInfoTicketBFCL/simple_python_328 (boardgame.get_info): Ticket to Ride, {rating, player count} either order, language omitted/English.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN24 655 OUT531

×

SimpleFindCardInDeckBFCL/simple_python_335 (find_card_in_deck): rank Queen, suit Hearts, deck omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN20 026 OUT231

×

SimpleGameListByYearBFCL/simple_python_342 (game_list.get_games): year 2019, multiplayer true, ESRB Everyone.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN55 070 OUT1 163

✓

SimpleGameWinnerByDateBFCL/simple_python_307 (game_result.get_winner): Lakers/Clippers either order, date 2021-01-28, venue omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN7 652 OUT102

×

SimpleHighestGameScoreBFCL/simple_python_349 (game_score.highest): Overwatch, PC, region Global or omitted.1 AI query 10 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN49 299 OUT1 492

×

SimpleMatchScheduleBFCL/simple_python_308 (sports.match_schedule): Man United, 5 matches, English Premier League or omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN20 151 OUT351

×

SimpleNoteDurationBFCL/simple_python_300 (music.calculate_note_duration): 440/880 Hz, tempo omitted or 120.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN15 812 OUT750

×

SimplePokerGameWinnerMultiBFCL/simple_python_337 (poker_game_winner): 4 players + per-player hands dict, type omitted/Texas Holdem.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN16 478 OUT1 033

×

SimpleSoccerPlayerStatsBFCL/simple_python_303 (soccer_stat.get_player_stats): Cristiano Ronaldo, 2019-2020, league omitted.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN20 166 OUT451

×

SimpleSportRankingPlayerBFCL/simple_python_316 (get_sport_ranking): Tennis, Serena Williams, gender omitted/all/female.1 AI query 4 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN20 113 OUT447

×

SimpleTeamRankRegularBFCL/simple_python_317 (get_team_rank): LA Lakers, NBA, 2021, type regular.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN20 247 OUT470

Tool Calling - Simple: Sports & Games (set 2) 1 passed10 failed

9.1%

IN307 404 OUT7 401

×

SimpleBoardGameInfoCatanBFCL/simple_python_330 (board_game_info): Catan, {average_review_rating, age_range}.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN37 155 OUT719

×

SimpleCricketBattingScoreBFCL/simple_python_306 (average_batting_score): Virat Kohli, 10 matches, T20 or omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 789 OUT425

×

SimpleFifaTeamRankingBFCL/simple_python_318 (get_team_ranking): Germany, 2021, gender men or omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 796 OUT529

✓

SimpleMatchResultsTwoTeamsBFCL/simple_python_325 (sports.match_results): Chicago Bulls vs Los Angeles Lakers, season omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 114 OUT80

×

SimplePokerDetermineWinnerTwoBFCL/simple_python_340 (card_games.poker_determine_winner): John vs Mike with 5-card hands (suit symbols).1 AI query 19 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 19

Failed

IN113 704 OUT3 271

×

SimpleShuffleAndDrawBFCL/simple_python_336 (cards.shuffle_and_draw): num_cards 3.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 660 OUT186

×

SimpleSoccerPerformanceBFCL/simple_python_305 (sports_stats.get_performance): Messi, La Liga, 2020-2021, {goals scored, assists made}.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 486 OUT564

×

SimpleTeamScheduleGamesBFCL/simple_python_327 (sports_team.get_schedule): Man United, 6 games, Premier League, location omitted.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 796 OUT356

×

SimpleThirdChordBFCL/simple_python_301 (get_third_chord): key C, type major or omitted.1 AI query 8 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN37 434 OUT575

×

SimpleTopPlayerBySportBFCL/simple_python_323 (sports_ranking.get_top_player): tennis, gender women.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 706 OUT290

×

SimpleVideoGamePlayerStatsBFCL/simple_python_343 (game_stats.fetch_player_statistics): Zelda, user Sam, platform Switch.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN14 764 OUT406

Tool Calling - Simple: Sports & Games (set 3) 2 passed7 failed

22.2%

IN1 098 264 OUT16 727

✓

SimpleCardDrawProbabilityBFCL/simple_python_338 (card_game_probability.calculate): total 52, desired 13 (hearts), cards_drawn 1 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 468 OUT139

×

SimpleCelebrityNetWorthBFCL/simple_python_313 (celebrity_net_worth.get): Messi, currency EUR/euro.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 111 OUT329

✓

SimpleDefenseRankingBFCL/simple_python_315 (get_defense_ranking): season 2021, top 1 or omitted.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 459 OUT80

×

SimpleFindAthleteBFCL/simple_python_311 (sports_db.find_athlete): Lebron James, sport Basketball, team omitted.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN33 387 OUT982

×

SimpleGameItemStatsBFCL/simple_python_344 (get_game_item_stats): BotW, Guardian Sword+, stat power/power rating.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN372 895 OUT5 021

×

SimpleMonopolyOddsBFCL/simple_python_329 (monopoly_odds_calculator): number 7, dice_number 2, dice_faces 6 or omitted.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 878 OUT243

×

SimpleNflPlayerRecordBFCL/simple_python_309 (nfl_data.player_record): Tom Brady, 2020, team omitted or Tampa Bay Buccaneers.1 AI query 42 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 42

Failed

IN273 834 OUT3 761

×

SimpleSportsRankingSeasonIntBFCL/simple_python_319 (sports_ranking): Man United, Premier League, season omitted or 2023.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN361 126 OUT4 633

×

SimpleTeamLatestScoreBFCL/simple_python_324 (team_score.get_latest): Los Angeles Lakers, include_opponent true.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 106 OUT1 539

Tool Calling - Simple: Sports & Games (set 4) 1 passed6 failed

14.3%

IN832 436 OUT13 049

×

SimpleCareerStatsBFCL/simple_python_310 (get_career_stats): LeBron James, team omitted.1 AI query 8 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN31 133 OUT965

×

SimpleCelebrityAchievementsBFCL/simple_python_314 (sports_celebrity.get_major_achievements): Messi, sports Football/Soccer or omitted, team all or omitted.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN351 188 OUT4 048

×

SimpleChessTopPlayersBFCL/simple_python_331 (board_game.chess.get_top_players): New York, min rating 2300, number_of_players 10 or omitted.1 AI query 40 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 40

Failed

IN245 304 OUT3 675

✓

SimpleDeckOddsSuitBFCL/simple_python_341 (deck_of_cards.odds): suit hearts, deck_type without_joker or normal.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN5 975 OUT147

×

SimpleGameValuationBFCL/simple_python_345 (game_valuation): Super Mario Bros., 1985, condition Like New/New.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN36 031 OUT863

×

SimpleTeamPositionDetailedBFCL/simple_python_320 (sports_ranking.get_team_position): Golden State Warriors, 2022-2023, detailed true.1 AI query 27 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 27

Failed

IN143 208 OUT2 794

×

SimpleTeamScoreWithStatsBFCL/simple_python_326 (get_team_score): Los Angeles Lakers, NBA, include_player_stats any.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN19 597 OUT557

Tool Calling - Simple: Sports & Games (set 5) 7 failed

0.0%

IN815 323 OUT13 226

×

SimpleChessRatingBFCL/simple_python_332 (chess.rating): Magnus Carlsen, variant classical or omitted.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN18 920 OUT454

×

SimpleCollectablesInSeasonBFCL/simple_python_346 (get_collectables_in_season): Animal Crossing: New Horizons, Spring, item_type all or omitted.1 AI query 10 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN39 542 OUT1 084

×

SimpleDetailedWeatherForecastBFCL/simple_python_333 (detailed_weather_forecast): London UK, 3 days, {high/low temp, humidity, precipitation}.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN33 150 OUT1 862

×

SimpleFullHouseProbabilityBFCL/simple_python_339 (poker_probability.full_house): defaults deck_size 52, hand_size 5.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 911 OUT218

×

SimplePlayerStatisticYearBFCL/simple_python_312 (player_statistic): Ronaldo, year 2021, team omitted.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 50

Failed

IN340 816 OUT4 381

×

SimpleSoccerLastMatchBFCL/simple_python_347 (soccer.get_last_match): Liverpool F.C., include_stats true.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN18 947 OUT439

×

SimpleSportsRankingSeasonStrBFCL/simple_python_321 (sports_ranking): Barcelona, La Liga, 2021.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN350 037 OUT4 788

Tool Calling - Simple: Sports & Games (set 6) 1 passed3 failed

25.0%

IN57 512 OUT1 523

×

SimpleBlackjackWinnerBFCL/simple_python_334 (blackjack.check_winner): player [A,10], dealer [10,9], ace_value 1.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN28 219 OUT714

✓

SimpleCreatePlayerProfileBFCL/simple_python_348 (create_player_profile): StarPlayer, class Mage, starting_level 5.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN5 061 OUT87

×

SimpleCurrentTeamRankingBFCL/simple_python_322 (sports_ranking.get_current): Liverpool, Premier League, season omitted or 2023-2024.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN10 545 OUT296

×

SimplePlayerLastGameStatsBFCL/simple_python_304 (player_stats.getLastGame): LeBron James, Lakers, metrics {Points, Rebounds}.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN13 687 OUT426

Tool Calling - Simple: Stats, Finance & ML (set 1) 8 failed

0.0%

IN565 870 OUT8 795

×

SimpleCardProbabilityBFCL/simple_python_112 (calculate_probability): total_outcomes=52, favorable_outcomes=4 (king from deck), round_to omitted or 2.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN20 915 OUT500

×

SimpleCompoundInterestNBFCL/simple_python_131 (calculate_compound_interest): principal=10000, rate=0.05, time=10, n=4.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN9 968 OUT435

×

SimpleInvestmentProfitBFCL/simple_python_134 (investment.predictProfit): investment_amount=5000, annual_return=0.07, years=5.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN38 306 OUT742

×

SimpleLightTravelTimeBFCL/simple_python_100 (light_travel_time): distance_in_light_years=4, speed_of_light omitted or default.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN338 795 OUT3 624

×

SimpleQuarterlyDividendBFCL/simple_python_128 (finance.calculate_quarterly_dividend_per_share): total_payout=50000000, outstanding_shares=100000000.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN34 154 OUT739

×

SimpleRandomForestTrainBFCL/simple_python_109 (random_forest.train): n_estimators=100, max_depth=5, data='my_data'.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 422 OUT305

×

SimpleStockPriceByDateBFCL/simple_python_142 (get_stock_price): company_name Amazon/AMZN, date=2022-03-11, exchange NASDAQ or omitted.1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN75 040 OUT1 470

×

SimpleTTestStatsBFCL/simple_python_118 (stats.t_test): array_1=[10,15,12,14,11], array_2=[18,16,17,20,22], alpha=0.05.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN35 270 OUT980

Tool Calling - Simple: Stats, Finance & ML (set 2) 8 failed

0.0%

IN356 312 OUT6 882

×

SimpleCalculateSpeedBFCL/simple_python_101 (calculate_speed): distance=450, time=20, to_unit=km/h.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN26 016 OUT465

×

SimpleCardProbabilitySingleBFCL/simple_python_116 (probabilities.calculate_single): total_outcomes=52, event_outcomes=4 (king), round omitted or 2.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 235 OUT192

×

SimpleCompoundInterestFreqBFCL/simple_python_136 (compound_interest): principal=10000, annual_rate=5.0, compounding_freq=monthly, time_in_years=5.1 AI query 21 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 21

Failed

IN111 295 OUT2 199

×

SimpleDiscountedCashFlowBFCL/simple_python_129 (calculate_discounted_cash_flow): coupon_payment=100, period=5, discount_rate=0.04, face_value omitted or 1000.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN49 652 OUT831

×

SimpleHousePriceBedroomsBFCL/simple_python_110 (predict_house_price): bedrooms=3, bathrooms=2, area=1800, location San Francisco.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 956 OUT346

×

SimpleStockPriceDaysBFCL/simple_python_143 (get_stock_price): company=AAPL, days=5, exchange NASDAQ or omitted.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN25 892 OUT492

×

SimpleStockReturnProjectedBFCL/simple_python_137 (calculate_stock_return): investment_amount=5000, annual_growth_rate=0.06, holding_period=5, dividends omitted or false.1 AI query 17 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 17

Failed

IN83 546 OUT1 417

×

SimpleTTestIndBFCL/simple_python_119 (hypothesis_testing.ttest_ind): sample1=[22,33,42,12,34], sample2=[23,45,44,14,38], significance_level=0.05.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN31 720 OUT940

Tool Calling - Simple: Stats, Finance & ML (set 3) 1 passed7 failed

12.5%

IN193 175 OUT4 265

×

SimpleCompoundedInterestAnnuallyBFCL/simple_python_145 (calculate_compounded_interest): principal=5000, interest_rate=0.05, period=10, compounding_frequency omitted or Annually.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN25 407 OUT708

×

SimpleDistanceEarthMoonBFCL/simple_python_102 (calculate_distance): body1=Earth, body2=Moon, unit mi/miles/mile.1 AI query 15 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN68 678 OUT1 147

✓

SimpleHeartRatioProbabilityBFCL/simple_python_117 (probability_of_event): success_outcomes=13 (hearts), total_outcomes=52, format_as_ratio=true.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 515 OUT164

×

SimpleHousePriceRoomsBFCL/simple_python_125 (predict_house_price): area=2500, rooms=5, year=1990, location San Francisco/SF.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 434 OUT427

×

SimpleMutualFundReturnBFCL/simple_python_139 (estimate_mutual_fund_return): yearly_yield=5.0, investment_amount=2000, years=3.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 630 OUT446

×

SimpleReturnOnEquityBFCL/simple_python_132 (calculate_return_on_equity): net_income=2000000, shareholder_equity=10000000, dividends_paid=200000.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 778 OUT481

×

SimpleStockPriceCurrentMultiBFCL/simple_python_149 (get_stock_price): company_names=[Apple, Microsoft] (or tickers AAPL, MSFT).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 301 OUT238

×

SimpleTwoSampleTTestRunBFCL/simple_python_120 (run_two_sample_ttest): group1=[3,4,5,6,4], group2=[7,8,9,8,7], equal_variance=true.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN26 432 OUT654

Tool Calling - Simple: Stats, Finance & ML (set 4) 2 passed6 failed

25.0%

IN131 906 OUT3 648

✓

SimpleBinomialProbDistBFCL/simple_python_114 (prob_dist.binomial): trials=10, successes=5, p omitted or 0.5.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN6 578 OUT226

×

SimpleFutureValueMonthlyBFCL/simple_python_133 (finance.predict_future_value): present_value=5000, annual_interest_rate=0.05, compounding_periods_per_year=12, time_years=3.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 764 OUT593

×

SimpleGeometryAreaTriangleBFCL/simple_python_104 (geometry.area_triangle): base=6, height=10, unit omitted or 'square meters'.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 004 OUT198

✓

SimpleMathPowerBFCL/simple_python_105 (math.power): base=3, exponent=4, mod omitted or 1.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 554 OUT67

×

SimpleNPVCashFlowsBFCL/simple_python_127 (calculate_NPV): cash_flows=[200,300,400,500], discount_rate=0.1, initial_investment=2000.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 902 OUT929

×

SimpleReturnOnInvestmentBFCL/simple_python_135 (calculate_return_on_investment): purchase_price=20, sale_price=25, dividend=2.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN25 617 OUT505

×

SimpleStockPriceDataTypeBFCL/simple_python_146 (stock_price): company Amazon/AMZN, days=3, data_type omitted or Close.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 216 OUT364

×

SimpleTwoSampleTTestFloatBFCL/simple_python_123 (hypothesis_testing.two_sample_t_test): group1=[12.4,15.6,11.2,18.9], group2=[10.5,9.8,15.2,13.8], alpha omitted or 0.05.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN18 271 OUT766

Tool Calling - Simple: Stats, Finance & ML (set 5) 1 passed7 failed

12.5%

IN579 173 OUT9 600

×

SimpleAreaUnderCurvePolyBFCL/simple_python_103 (mathematics.calculate_area_under_curve): polynomial=[3.0,2.0,-4.0], limits=[-1.0,2.0].1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN21 722 OUT582

×

SimpleBinomialCalcBFCL/simple_python_115 (calculate_binomial_probability): number_of_trials=8, number_of_successes=5, probability_of_success omitted or 0.5.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 836 OUT388

✓

SimpleCalculateBmiBFCL/simple_python_107 (calculate_bmi): weight=70, height=175, system omitted or metric.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN6 656 OUT170

×

SimpleFutureValueInvestmentBFCL/simple_python_148 (finance.calculate_future_value): initial_investment=20000, rate_of_return=0.08, years=5, contribution omitted or 0.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN49 568 OUT1 035

×

SimpleLinearRegressionBFCL/simple_python_108 (run_linear_regression): predictors=[Age,Income,Education], target=Purchase_Amount, standardize=true.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 731 OUT372

×

SimpleNPVInvestmentBFCL/simple_python_130 (finance_calculator.npv): cash_flows=[-50000,10000,15000,20000,25000,30000], discount_rate=0.08, years omitted or empty.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN437 949 OUT6 376

×

SimpleStockPricesDurationBFCL/simple_python_147 (get_stock_prices): companies=[Microsoft,Google], duration='2 weeks'.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN17 763 OUT270

×

SimpleTTestDatasetsBFCL/simple_python_124 (t_test): dataset_A=[12,24,36], dataset_B=[15,30,45], alpha omitted or 0.05.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN13 948 OUT407

Tool Calling - Simple: Stats, Finance & ML (set 6) 2 passed8 failed

20.0%

IN957 307 OUT12 550

×

SimpleBinomialCoin100BFCL/simple_python_121 (calc_binomial_prob): num_trials=100, num_success=60, prob_success=0.5.1 AI query 15 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN74 742 OUT1 307

×

SimpleCagrBFCL/simple_python_140 (calculate_cagr): initial_value=2000, final_value=3000, period_in_years=4.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 264 OUT651

×

SimpleChiSquaredTestBFCL/simple_python_122 (chi_squared_test): table=[[10,20],[30,40]], alpha omitted or 0.05.1 AI query 8 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN36 989 OUT538

✓

SimpleDiceRollBFCL/simple_python_113 (probability.dice_roll): desired_number=6, number_of_rolls=2, die_sides omitted or 6.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 134 OUT167

×

SimpleMarketPerformanceBFCL/simple_python_144 (market_performance.get_data): indexes=[S&P 500, Dow Jones], days=5, detailed omitted or true/false.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN389 397 OUT4 571

×

SimpleMetalPriceBFCL/simple_python_141 (get_metal_price): metal=Gold, measure=ounce.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN365 062 OUT3 942

×

SimpleNormalVariateBFCL/simple_python_111 (random.normalvariate): mu=0, sigma=1.1 AI query 2 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN10 844 OUT240

×

SimplePortfolioFutureValueBFCL/simple_python_138 (portfolio_future_value): stock=X, invested_amount=5000, expected_annual_return=0.05, years=7.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN23 381 OUT580

✓

SimpleRandomForestClassifierBFCL/simple_python_106 (train_random_forest_classifier): dataset=your_dataset_name, max_depth=5, n_estimators=100.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN7 160 OUT162

×

SimpleRSquaredBFCL/simple_python_126 (linear_regression.get_r_squared): dataset_path=C:/data/cars.csv, independent_variables=[engine_size,fuel_economy], dependent_variable=car_price.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN19 334 OUT392

Tool Calling - Tool Selection: Area (Triangle/Circle) 1 failed

0.0%

IN18 329 OUT665

×

SelectTriangleAreaHeronToolBFCL/multiple_1 (math.triangle_area_heron): triangle area from three side lengths (Heron).1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN18 329 OUT665

Tool Calling - Tool Selection: Art & Music 9 failed

0.0%

IN2 112 428 OUT19 580

×

SelectAnalyzeStructureToolBFCL/multiple_162 (analyze_structure): dynamic structural analysis of specific floors.1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN72 781 OUT923

×

SelectConcertSearchToolBFCL/multiple_166 (concert.search): concert search by genre/location/date/price.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN50 505 OUT844

×

SelectGuitarPriceToolBFCL/multiple_165 (guitar_price.find): guitar price by model/condition/location (not instrument_price.get).1 AI query 18 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 18

Failed

IN136 454 OUT1 539

×

SelectInstrumentPriceToolBFCL/multiple_164 (instrument_price.get): instrument price by brand/model/finish (not guitar_price.find).1 AI query 50 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN594 959 OUT4 775

×

SelectMelodyGeneratorToolBFCL/multiple_167 (music_generator.generate_melody): melody generation with key/note/length/tempo. Tightened per methodology: prompt explicitly gives "120 beats per minute", so tempo is required exactly 120 (ground truth also accepts omission, which equals the schema default).1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 18

Failed

IN138 001 OUT1 715

×

SelectMetropolitanArtworksToolBFCL/multiple_163 (metropolitan_museum.get_top_artworks): top artworks sorted by popularity.1 AI query 28 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 28

Failed

IN228 899 OUT2 499

×

SelectMusicalScaleToolBFCL/multiple_169 (musical_scale): musical scale for a key.1 AI query 41 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 41

Failed

IN384 015 OUT3 161

×

SelectSculptureInfoToolBFCL/multiple_160 (get_sculpture_info): sculpture info by artist with detail.1 AI query 45 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 45

Failed

IN420 185 OUT3 226

×

SelectSongLyricsToolBFCL/multiple_168 (get_song_lyrics): lyrics by song title/artist.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN86 629 OUT898

Tool Calling - Tool Selection: Calculators & Directories 8 failed

0.0%

IN942 412 OUT10 381

×

SelectCalculateGcdToolBFCL/multiple_35 (calculate_gcd): greatest common divisor, not lcm.1 AI query 29 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 29

Failed

IN280 168 OUT2 747

×

SelectCurrencyGetRateToolBFCL/multiple_17 (currency_conversion.get_rate): rate at a specific date, not convert/latest-rate.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN30 587 OUT497

×

SelectGeodistanceToolBFCL/multiple_65 (geodistance.find): distance between cities, not timezone difference/flights.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN59 861 OUT763

×

SelectLibrarySearchBookToolBFCL/multiple_38 (library.search_book): search for a book, not reserve one.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN431 649 OUT4 094

×

SelectLinearRegressionToolBFCL/multiple_53 (linear_regression): linear regression, not random forest.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN48 971 OUT805

×

SelectMagneticFieldCalculateToolBFCL/multiple_41 (magnetic_field.calculate): magnetic field via Ampere's law, not electric field/force.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN30 451 OUT571

×

SelectSolveQuadraticEquationToolBFCL/multiple_28 (solve.quadratic_equation): solve a quadratic, not rgb/string-reverse.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN30 427 OUT422

×

SelectStockForecastToolBFCL/multiple_55 (stock_forecast): stock price prediction, not weather forecast.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN30 298 OUT482

Tool Calling - Tool Selection: Finance & Legal 1 passed9 failed

10.0%

IN2 782 753 OUT24 185

×

SelectCagrCalculationToolBFCL/multiple_130 (calculate_cagr): compound annual growth rate from initial/final/years, not other finance tools.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN27 547 OUT350

×

SelectCaseInfoToolBFCL/multiple_135 (get_case_info): retrieve case info (victim) by docket/court.1 AI query 30 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 30

Failed

IN272 005 OUT2 561

×

SelectCrimeRateToolBFCL/multiple_136 (get_crime_rate): crime rate by city/state/type/year.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN636 330 OUT5 034

×

SelectCrimeRecordToolBFCL/multiple_134 (crime_record.get_record): detailed crime record by case number/county.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN79 723 OUT799

✓

SelectFutureValueToolBFCL/multiple_132 (finance.calculate_future_value): future value from rate/initial/years, not compound_interest.1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN13 179 OUT92

×

SelectLawsuitDetailsToolBFCL/multiple_139 (lawsuit_details.find): patent lawsuit details by company/year/type.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN528 744 OUT4 167

×

SelectLawsuitSearchToolBFCL/multiple_137 (lawsuit_search): lawsuit search by company/date/location/status. Tightened per methodology: prompt explicitly asks for "ongoing" lawsuits, so status is required exactly "ongoing" (ground truth also accepts omission, but an explicitly-stated value must be passed).1 AI query 36 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 36

Failed

IN346 326 OUT3 303

×

SelectLegalCaseFetchToolBFCL/multiple_138 (legal_case.fetch): fetch detailed legal case by id.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN524 712 OUT3 981

×

SelectMarketPerformanceToolBFCL/multiple_131 (market_performance.get_data): index performance over N days.1 AI query 32 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 32

Failed

IN303 650 OUT3 295

×

SelectMutualFundBalanceToolBFCL/multiple_133 (calculate_mutual_fund_balance): mutual fund balance from investment/yield/years.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN50 537 OUT603

Tool Calling - Tool Selection: Finance & Markets 8 failed

0.0%

IN860 722 OUT12 007

×

SelectAvgClosingPriceToolBFCL/multiple_56 (avg_closing_price): average closing price, not volume/total revenue.1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN76 621 OUT1 016

×

SelectCompoundInterestCalcToolBFCL/multiple_49 (calc_Compound_Interest): compound interest, not future value/simple interest.1 AI query 24 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 24

Failed

IN174 880 OUT2 462

×

SelectCurrencyConversionToolBFCL/multiple_52 (currency_conversion): currency conversion, not unit conversion.1 AI query 29 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 29

Failed

IN218 665 OUT2 572

×

SelectDividendDataToolBFCL/multiple_54 (corporate_finance.dividend_data): dividend history, not generic stock market data.1 AI query 13 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 13

Failed

IN86 272 OUT1 604

×

SelectHousePriceForecastToolBFCL/multiple_50 (house_price_forecast): house price forecast, not weather/stock forecast.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN44 251 OUT515

×

SelectPropertyDepreciationToolBFCL/multiple_14 (finance.property_depreciation): depreciation, not loan/inflation.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN64 357 OUT1 189

×

SelectRevenueForecastToolBFCL/multiple_13 (corporate_finance.revenue_forecast): revenue forecast, not product price.1 AI query 24 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 24

Failed

IN174 205 OUT2 234

×

SelectYoyGrowthRateToolBFCL/multiple_12 (corporate_finance.calculate_YOY_growth_rate): YOY growth, not ROE/ROA.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN21 471 OUT415

Tool Calling - Tool Selection: Food & Travel 7 failed

0.0%

IN1 416 251 OUT15 528

×

SelectCurrencyConvertToolBFCL/multiple_95 (currency_exchange.convert): currency conversion, not unit_conversion. live_conversion is required true: prompt says "using current exchange rate" (ground truth = [true]).1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN24 129 OUT454

×

SelectDrivingDistanceToolBFCL/multiple_104 (get_shortest_driving_distance): driving distance, not instrument price / biology / scientist lookup.1 AI query 11 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN64 351 OUT845

×

SelectHotelBookToolBFCL/multiple_93 (hotel.book): book a hotel room, not car.rental.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN15 715 OUT189

×

SelectRecipeFindToolBFCL/multiple_91 (recipe.find): recipe by main ingredient + limit, not restaurant.find.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN415 171 OUT4 049

×

SelectRecipeSearchToolBFCL/multiple_89 (recipe_search): find a recipe from ingredients, not restaurant_search/ingredient_replace.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN435 979 OUT4 859

×

SelectRestaurantForGroupToolBFCL/multiple_90 (restaurant.find_group): restaurant for a group, not events.find_event.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN436 853 OUT4 778

×

SelectWalmartVeganToolBFCL/multiple_92 (walmart.vegan_products): the named store is Walmart, not Safeway/Whole Foods.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN24 053 OUT354

Tool Calling - Tool Selection: Games, Food & Utilities 10 failed

0.0%

IN2 055 075 OUT20 178

×

SelectBestGroceryStoreToolBFCL/multiple_188 (grocery_store.find_best): best grocery store by rating/products.1 AI query 25 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 25

Failed

IN234 746 OUT3 481

×

SelectClosestRestaurantToolBFCL/multiple_185 (restaurant_search.find_closest): closest restaurant by cuisine/amenities.1 AI query 21 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 21

Failed

IN187 340 OUT1 974

×

SelectFindRecipeToolBFCL/multiple_186 (find_recipe): recipe by restrictions/type/time.1 AI query 31 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 31

Failed

IN295 821 OUT3 058

×

SelectGamePlayerStatsToolBFCL/multiple_180 (game_stats.fetch_player_statistics): player stats by game/username/platform.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN35 500 OUT293

×

SelectMultiplayerGameFinderToolBFCL/multiple_182 (multiplayer_game_finder): multiplayer game by platform/rating.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN42 988 OUT519

×

SelectRecipeCaloriesToolBFCL/multiple_183 (recipe_info.get_calories): calories of a recipe from a website.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN543 637 OUT4 230

×

SelectRecipeSearchToolBFCL/multiple_184 (recipe_search): recipe by restriction/ingredients/servings.1 AI query 4 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN39 410 OUT654

×

SelectSoccerLastMatchToolBFCL/multiple_181 (soccer.get_last_match): last match with stats for a club.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN42 961 OUT387

×

SelectTimezoneConvertToolBFCL/multiple_189 (timezone.convert): convert a time between timezones.1 AI query 50 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN549 332 OUT4 737

×

SelectWholeFoodsPriceToolBFCL/multiple_187 (whole_foods.check_price): price of items at a Whole Foods location.1 AI query 10 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN83 340 OUT845

Tool Calling - Tool Selection: Geometry, Finance & Services 7 failed

0.0%

IN1 070 658 OUT14 223

×

SelectAirQualityForecastToolBFCL/multiple_63 (air_quality_forecast): air quality, not weather/news.1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 18

Failed

IN102 615 OUT1 290

×

SelectFinancialCompoundInterestToolBFCL/multiple_57 (financial.compound_interest): compound interest, not simple interest.1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 18

Failed

IN106 928 OUT1 682

×

SelectGeometryRectangleToolBFCL/multiple_31 (geometry_rectangle.calculate): rectangle area+perimeter, not square/circle.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN396 271 OUT3 978

×

SelectLawyerFinderToolBFCL/multiple_59 (lawyer_finder): find a lawyer by specialization, not rent a car.1 AI query 7 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 7

Failed

IN36 969 OUT754

×

SelectLibrarySearchBooksToolBFCL/multiple_68 (library.search_books): search a specific library, not Google Books/Open Library.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN17 283 OUT203

×

SelectMapsShortestPathToolBFCL/multiple_27 (maps.shortest_path): shortest path with transport mode, not route-time estimation.1 AI query 16 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 16

Failed

IN91 413 OUT1 602

×

SelectTrafficEstimateToolBFCL/multiple_66 (traffic_estimate): traffic for a time period, not raw distance/weather.1 AI query 42 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 42

Failed

IN319 179 OUT4 714

Tool Calling - Tool Selection: History & Knowledge 1 passed9 failed

10.0%

IN3 457 467 OUT25 380

×

SelectDiscovererToolBFCL/multiple_155 (get_discoverer): discoverer of a discovery, with detail.1 AI query 54 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN965 832 OUT4 282

×

SelectEarliestReferenceToolBFCL/multiple_157 (get_earliest_reference): earliest reference of a person from a source.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN661 337 OUT4 724

×

SelectEventDateToolBFCL/multiple_153 (get_event_date): date of a historical event.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN20 518 OUT119

×

SelectHistoricalContribToolBFCL/multiple_156 (historical_contrib.get_contrib): scientist's contribution on a date.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN77 676 OUT798

×

SelectHistoryKeyEventsToolBFCL/multiple_152 (history.get_key_events): key war events by country/years.1 AI query 44 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 44

Failed

IN478 278 OUT5 642

×

SelectMostFollowedToolBFCL/multiple_151 (social_media_analytics.most_followed): most-followed by topic/sub-topics.1 AI query 50 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN542 184 OUT4 498

×

SelectNeuronalActivityToolBFCL/multiple_150 (calculate_neuronal_activity): neuronal activity from synaptic rate/weight/decay.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN74 320 OUT719

×

SelectPaintNeededToolBFCL/multiple_159 (calculate_paint_needed): paint quantity for a wall.1 AI query 27 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 27

Failed

IN247 361 OUT2 084

×

SelectPapalBiographyToolBFCL/multiple_158 (religious_history.get_papal_biography): papal biography with contributions.1 AI query 38 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 38

Failed

IN376 360 OUT2 431

✓

SelectUsPresidentToolBFCL/multiple_154 (US_president.in_year): president in a given year. Tightened per methodology: prompt explicitly asks for the "full name", so full_name is required true (ground truth also accepts omission, but an explicitly-requested option must be passed).1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 601 OUT83

Tool Calling - Tool Selection: History, Demographics & Art 8 failed2 skipped

0.0%

IN964 550 OUT12 429

×

SelectArtworkPriceToolBFCL/multiple_74 (art_auction.fetch_artwork_price): artwork price, not library book search. Tightened per methodology: prompt explicitly asks "on all auction platforms", so require platform == "all" (ground truth also accepts omission, but an explicitly-given value must pass).1 AI query 46 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 46

Failed

IN93 440 OUT4 099

×

SelectArtworkSearchToolBFCL/multiple_77 (artwork_search.find): artwork search, not tourist_attraction/park search.1 AI query 32 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 32

Failed

IN256 873 OUT2 826

–

SelectCustomSculptureToolBFCL/multiple_76 (sculpture.create_custom): custom sculpture (material), not painting (color).0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Skipped: Ambiguous action-vs-question item (item flaw, not model behavior): the prompt mixes an order ('I want to order a custom bronze sculpture of a horse') with a question ('What material options are available?'), so a competent model may either call sculpture_create_custom or answer the question in text - not deterministic. Kept for traceability.

Skipped

IN0 OUT0

×

SelectEuropeanMonarchsToolBFCL/multiple_70 (european_history.get_monarchs): monarch lookup, not events/culture.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN33 814 OUT519

×

SelectExhibitionInfoToolBFCL/multiple_79 (exhibition_info): museum exhibitions, not restaurant_info.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN33 882 OUT378

–

SelectMuseumInfoToolBFCL/multiple_78 (museum_info): museum info with a feature list, not generic tourist_spot_info.0 AI queries 0 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Skipped: Ambiguous single-call item (item flaw, not model behavior): a dedicated exhibition_info tool coexists with museum_info, so the 'exhibitions' part can reasonably route either to museum_info's features or to a separate exhibition_info call; a competent model may make one or two calls - not deterministic. Kept for traceability.

Skipped

IN0 OUT0

×

SelectPaintColorTrendsToolBFCL/multiple_75 (paint_color.trends): paint color trend, not weather/house-price trends.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN474 694 OUT3 651

×

SelectPopulationByCategoryToolBFCL/multiple_71 (get_population): population by category, not bureau statistics/demographics.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN22 092 OUT439

×

SelectReligionOriginToolBFCL/multiple_73 (religion.get_origin): founder/origin, not core beliefs.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN21 939 OUT202

×

SelectStatePopulationToolBFCL/multiple_72 (us_history.population_by_state_year): state population, not GDP.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN27 816 OUT315

Tool Calling - Tool Selection: Isolated Collisions: Exhibition, Hotel & Currency 3 failed

0.0%

IN85 568 OUT1 492

×

SelectBookLuxuryHotelToolBFCL/multiple_191 (book_hotel): luxury room with view, stay_duration variant.1 AI query 3 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN16 589 OUT283

×

SelectCurrencyConversionToolBFCL/multiple_192 (currency_conversion.convert): convert an amount between currencies.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN22 619 OUT407

×

SelectExhibitionToolBFCL/multiple_161 (find_exhibition): top-rated sculpture exhibition by location/art_form.1 AI query 9 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN46 360 OUT802

Tool Calling - Tool Selection: Knowledge & Media 1 passed9 failed

10.0%

IN819 953 OUT9 745

×

SelectBattleDetailsToolBFCL/multiple_18 (european_history.battle_details): battle details, not war/leader info.1 AI query 26 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 26

Failed

IN206 059 OUT1 676

×

SelectDatabaseModifyColumnsToolBFCL/multiple_10 (database.modify_columns): delete columns, not create a backup.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN47 792 OUT503

×

SelectFiveFactorModelToolBFCL/multiple_69 (five_factor_model.analyse): Big Five personality, not MBTI.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN30 680 OUT446

×

SelectGameRewardsToolBFCL/multiple_26 (game_rewards.get): rewards, not scores/missions list.1 AI query 27 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 27

Failed

IN193 757 OUT2 680

×

SelectGenerateSoundWaveToolBFCL/multiple_21 (generate_sound_wave): generate a sound, not play a file.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN38 029 OUT381

✓

SelectMostPointsSingleGameToolBFCL/multiple_22 (sports_data.basketball.most_points_single_game): single-game record, not season/career.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN11 751 OUT279

×

SelectPlayerStatsToolBFCL/multiple_23 (basketball.player_stats.get): player stats, not game/team stats.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN30 685 OUT420

×

SelectReligionSchismsToolBFCL/multiple_19 (religion_history.get_schisms): schisms, not councils/reformations.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN30 310 OUT470

×

SelectSculpturePriceToolBFCL/multiple_20 (sculpture_price.calculate): commission price, not sculptor info/availability.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN17 810 OUT278

×

SelectVideoGameStorePriceToolBFCL/multiple_25 (video_games.store_price): store price, not store currency/on-sale.1 AI query 26 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 26

Failed

IN213 080 OUT2 612

Tool Calling - Tool Selection: Legal, Environment & Services 10 failed

0.0%

IN2 945 820 OUT27 186

×

SelectAirQualityToolBFCL/multiple_144 (air_quality): AQI by location/date.1 AI query 40 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 40

Failed

IN371 050 OUT3 618

×

SelectEmissionsToolBFCL/multiple_145 (calculate_emissions): annual CO2 from distance/fuel/efficiency.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN511 228 OUT4 645

×

SelectHumidityForecastToolBFCL/multiple_142 (weather.humidity_forecast): humidity forecast by location/days.1 AI query 30 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 30

Failed

IN254 613 OUT2 073

×

SelectLawsuitCheckCaseToolBFCL/multiple_141 (lawsuit.check_case): verify a case's closed status by id.1 AI query 20 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 20

Failed

IN156 655 OUT1 410

×

SelectLawsuitsSearchToolBFCL/multiple_140 (lawsuits_search): lawsuits by company/location/year.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN498 752 OUT3 931

×

SelectMapDirectionsToolBFCL/multiple_147 (map_service.get_directions): directions avoiding features.1 AI query 27 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 27

Failed

IN231 052 OUT3 046

×

SelectRestaurantFindNearbyToolBFCL/multiple_146 (restaurant.find_nearby): nearby restaurants by cuisine/distance.1 AI query 30 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 30

Failed

IN258 621 OUT2 595

×

SelectSentimentAnalysisToolBFCL/multiple_149 (sentiment_analysis): sentiment of a review text.1 AI query 15 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN114 117 OUT1 213

×

SelectSlopeGradientToolBFCL/multiple_143 (calculate_slope_gradient): slope gradient between two coordinates. Tightened per methodology: prompt explicitly asks for the gradient "in degree", so unit is required exactly "degree" (ground truth also accepts omission, which equals the schema default).1 AI query 6 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN45 766 OUT755

×

SelectStockInfoToolBFCL/multiple_148 (get_stock_info): detailed stock information by company.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN503 966 OUT3 900

Tool Calling - Tool Selection: Local Services & Weather 9 failed

0.0%

IN1 732 470 OUT19 161

×

SelectHumidityTemperatureForecastToolBFCL/multiple_60 (humidity_temperature_forecast): humidity/temperature, not events/movies.1 AI query 19 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 19

Failed

IN122 275 OUT1 314

×

SelectLandscapeArchitectToolBFCL/multiple_61 (landscape_architect.find_specialty): landscape architect, not home renovation.1 AI query 17 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 17

Failed

IN109 357 OUT1 260

×

SelectLawyerSearchToolBFCL/multiple_58 (lawyer.search): find a lawyer, not a doctor.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN14 599 OUT207

×

SelectLibraryFindNearbyToolBFCL/multiple_48 (library.find_nearby): find a library, not a store.1 AI query 48 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 48

Failed

IN427 878 OUT4 359

×

SelectNatureParkToolBFCL/multiple_62 (nature_park.find_nearby): find a nature park, not a restaurant.1 AI query 44 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 44

Failed

IN376 033 OUT3 996

×

SelectRideHailingToolBFCL/multiple_39 (ride_hailing.get_rides): find a ride, not a grocery delivery.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN54 085 OUT877

×

SelectRoutePlannerToolBFCL/multiple_24 (route_planner.calculate_route): plan a route, not chess-club lookup.1 AI query 42 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 42

Failed

IN345 671 OUT3 764

×

SelectTranslateToolBFCL/multiple_67 (translate): translate text, not sentiment/word-count.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN14 635 OUT174

×

SelectUvIndexToolBFCL/multiple_64 (uv_index.get_future): UV index, not rainfall/snowfall.1 AI query 35 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 35

Failed

IN267 937 OUT3 210

Tool Calling - Tool Selection: Mathematics 2 passed6 failed

25.0%

IN1 144 609 OUT12 918

×

SelectCalculateIntegralToolBFCL/multiple_33 (calculate_integral): definite integral, not derivative.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN24 525 OUT384

×

SelectCosineSimilarityToolBFCL/multiple_47 (cosine_similarity.calculate): cosine similarity, not correlation.1 AI query 15 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN93 597 OUT1 875

✓

SelectDiceRollProbabilityToolBFCL/multiple_51 (dice_roll_probability): dice sum probability, not coin/card.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 411 OUT154

×

SelectFunctionsIntersectToolBFCL/multiple_29 (functions.intersect): intersection of two functions, not zeros of one.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN464 454 OUT4 449

×

SelectMathLcmToolBFCL/multiple_34 (math.lcm): least common multiple, not gcd/sqrt.1 AI query 13 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 13

Failed

IN77 583 OUT749

×

SelectQuadraticRootsToolBFCL/multiple_11 (math_roots.quadratic): quadratic roots from coefficients, not cubic/polynomial.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN29 887 OUT604

✓

SelectRectangleAreaToolBFCL/multiple_30 (rectangle.area): rectangle area, not circle/triangle area.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 401 OUT65

×

SelectSortListToolBFCL/multiple_46 (sort_list): sort a list, not filter/sum.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN435 751 OUT4 638

Tool Calling - Tool Selection: Mathematics & Geometry 2 passed5 failed

28.6%

IN151 628 OUT2 055

✓

SelectCircleAreaToolBFCL/multiple_97 (geometry.area_circle): circle area, not plot_sine_wave.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 030 OUT106

×

SelectCircumferenceToolBFCL/multiple_98 (geometry.circumference): circle circumference, not the reference/time/melody tools.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN20 479 OUT262

×

SelectDatabaseQueryToolBFCL/multiple_119 (database.query): structured DB query with conditions, not the score/ROI/probability distractors.1 AI query 2 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN15 358 OUT276

×

SelectDerivativeToolBFCL/multiple_99 (calculus.derivative): derivative at a point, not get_personality_traits.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN37 653 OUT504

×

SelectHcfToolBFCL/multiple_100 (math.hcf): highest common factor, not the scale-progression distractor.1 AI query 6 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN37 460 OUT465

×

SelectSolveQuadraticToolBFCL/multiple_96 (solve_quadratic_equation): solve quadratic, not stock/portfolio tools.1 AI query 3 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN20 607 OUT376

✓

SelectTriangleAreaToolBFCL/multiple_121 (geometry.area_triangle): triangle area, not map directions / invention lookup.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN10 041 OUT66

Tool Calling - Tool Selection: Mixed Domains 7 failed1 skipped

0.0%

IN536 355 OUT8 505

×

SelectAverageGradeToolBFCL/multiple_9 (calculate_average): selects average, not standard_deviation/highest_grade.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN61 222 OUT851

×

SelectCapacitanceToolBFCL/multiple_6 (capacitance_calculator.calculate): selects capacitance, not resistance/magnetic_field. Tightened per methodology: prompt explicitly states "dielectric constant K is 1.0", so K is required exactly 1.0 (ground truth also accepts omission, but an explicitly-given value must be passed).1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN31 291 OUT626

×

SelectCapitalToolBFCL/multiple_2 (country_info.capital): selects capital lookup, not largest_city/population.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN19 949 OUT146

×

SelectDisplacementToolBFCL/multiple_4 (kinematics.calculate_displacement): selects displacement, not final_speed.1 AI query 15 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 15

Failed

IN96 389 OUT1 427

×

SelectEuclideanDistanceToolBFCL/multiple_3 (EuclideanDistance.calculate): selects distance, not angleToXAxis.1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 18

Failed

IN118 336 OUT1 658

×

SelectRealEstateSearchToolBFCL/multiple_8 (realestate.find_properties): property search with budget, not property_valuation.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN63 211 OUT1 330

×

SelectWeatherByCoordinatesDateToolBFCL/multiple_5 (weather.get_by_coordinates_date): historical-by-coordinates, not by-city/forecast.1 AI query 21 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 21

Failed

IN145 957 OUT2 467

–

SelectWildlifeGrowthToolBFCL/multiple_7 (wildlife_population.assess_growth): population growth, not ecological_impact.0 AI queries 0 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Skipped: Ambiguous single-call item (item flaw, not model behavior): the prompt asks for BOTH population growth AND impact on woodland, and two distinct tools genuinely fit (wildlife_population_assess_growth + ecological_impact_analyze); a competent human would reasonably make two calls, so a single-call ground truth is unfair. Kept for traceability.

Skipped

IN0 OUT0

Tool Calling - Tool Selection: Music, Sports & Games 1 passed8 failed

11.1%

IN787 483 OUT8 885

×

SelectBoardGameRecommendToolBFCL/multiple_86 (BoardGameGeek.recommend): BoardGameGeek store, not AmazonGameStore.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN20 531 OUT524

×

SelectConcertTicketToolBFCL/multiple_81 (concert.book_ticket): concert ticket, not festival ticket.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN56 426 OUT765

×

SelectGameUpdateToolBFCL/multiple_87 (games.update.find): game update/patch, not reviews/price.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN26 060 OUT399

✓

SelectMusicGenerateToolBFCL/multiple_82 (music.generate): musical piece from key/tempo, not raw audio.generate.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN10 012 OUT73

×

SelectMusicShopToolBFCL/multiple_80 (music_shop.find_nearby): music shop with lessons, not gym.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN31 777 OUT404

×

SelectPlayerCountToolBFCL/multiple_88 (video_games.get_player_count): active player count, not sales.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN35 402 OUT487

×

SelectPlayerGoalsToolBFCL/multiple_83 (player_stats.get_all_time_goals): per-player goals, not team/league top scorer.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN451 555 OUT4 191

×

SelectSoccerScoresToolBFCL/multiple_85 (soccer_scores.get_scores): soccer scores, not basketball.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN43 375 OUT598

×

SelectTopGoalScorersToolBFCL/multiple_84 (getTopGoalScorers): top goal scorers, not top assists.1 AI query 17 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 17

Failed

IN112 345 OUT1 444

Tool Calling - Tool Selection: Physical Sciences 10 failed

0.0%

IN1 409 630 OUT15 815

×

SelectBiologicalEnergyToolBFCL/multiple_43 (biological.calc_energy): energy from moles of substance, not biomass/work.1 AI query 17 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 17

Failed

IN123 586 OUT1 350

×

SelectBiotSavartLawToolBFCL/multiple_40 (electromagnetism.biot_savart_law): magnetic field via Biot-Savart, not Ampere's law.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN23 091 OUT352

×

SelectConeVolumeToolBFCL/multiple_32 (geometry.calculate_cone_volume): cone volume, not cone mass.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN17 089 OUT346

×

SelectFinalTemperatureToolBFCL/multiple_42 (calculate_final_temperature): mixing two gas samples, not mass calculation.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN42 339 OUT723

×

SelectFinalVelocityToolBFCL/multiple_37 (kinematics.final_velocity): final velocity from initial velocity + time, not wave velocity/distance.1 AI query 18 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 18

Failed

IN130 107 OUT1 400

×

SelectGeologyEraToolBFCL/multiple_45 (geology.get_era): geological era date, not historical event date.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN474 264 OUT4 262

×

SelectPopulationGeneticsNeToolBFCL/multiple_16 (population_genetics.calculate_ne): effective population size, not range-shift/carrying-capacity.1 AI query 6 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 6

Failed

IN42 243 OUT1 617

×

SelectSolarFarmPotentialToolBFCL/multiple_15 (solarFarm.potential): solar (not wind) energy potential from coordinates + panel area.1 AI query 8 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 8

Failed

IN56 473 OUT1 125

×

SelectSpeedFromRestToolBFCL/multiple_36 (kinematics.calculate_speed_from_rest): speed from rest over a distance, not acceleration.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN23 109 OUT340

×

SelectWeightInSpaceToolBFCL/multiple_44 (calculate.weight_in_space): weight on a planet, not unit/currency conversion.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN477 329 OUT4 300

Tool Calling - Tool Selection: Physics & Biology 13 failed

0.0%

IN3 436 394 OUT26 499

×

SelectCellProteinsToolBFCL/multiple_109 (cellbio.get_proteins): proteins by cell compartment, not find_exhibition / mutation lookup.1 AI query 23 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 23

Failed

IN218 886 OUT2 089

×

SelectDensityToolBFCL/multiple_107 (calculate_density): mass/volume density, not mix_paint_color.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN563 857 OUT3 901

×

SelectDisplacementToolBFCL/multiple_102 (calculate_displacement): kinematic displacement, not poker/musical distractors.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN38 560 OUT696

×

SelectElectricFieldToolBFCL/multiple_106 (calculate_electric_field_strength): electric field, not paint/cooking/group distractors.1 AI query 42 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 42

Failed

IN453 425 OUT3 672

×

SelectEvolutionPredictionToolBFCL/multiple_114 (prediction.evolution): evolutionary-rate prediction, not modify_painting / calculate_probability.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN46 717 OUT429

×

SelectFinalSpeedToolBFCL/multiple_103 (calculate_final_speed): free-fall final speed, not chess/court/event distractors.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN30 455 OUT279

×

SelectFitnessToolBFCL/multiple_113 (calculate_fitness): evolutionary fitness from trait values/contributions, not chess/walmart/lawyer distractors. Percentages map to decimal contributions [0.4, 0.6].1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN46 921 OUT436

×

SelectForestGrowthToolBFCL/multiple_112 (forest_growth_forecast): forest growth forecast, not get_scientist_for_discovery. include_human_impact is required true: prompt says "including human impact" (ground truth = [true]).1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN580 126 OUT4 088

×

SelectGenotypeFrequencyToolBFCL/multiple_111 (calculate_genotype_frequency): genotype frequency, not factorial / find_flute.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.genotype == "AA" Values: args.genotype = "homozygous dominant"

Failed

IN15 009 OUT141

×

SelectHeatCapacityToolBFCL/multiple_108 (calc_heat_capacity): gas heat capacity, not discounted_cash_flow.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN578 086 OUT4 456

×

SelectLightTravelTimeToolBFCL/multiple_120 (light_travel_time): light travel time, not grocery/hotel/exchange distractors.1 AI query 50 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN556 003 OUT3 627

×

SelectMagneticFieldToolBFCL/multiple_105 (calculate_magnetic_field): magnetic field of a loop, not concert/lawsuit distractors.1 AI query 2 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 2

Failed

IN22 679 OUT346

×

SelectMutationTypeToolBFCL/multiple_110 (mutation_type.find): SNP mutation type, not get_collectables_in_season.1 AI query 29 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 29

Failed

IN285 670 OUT2 339

Tool Calling - Tool Selection: Sports & Card Games 1 passed9 failed

10.0%

IN2 905 572 OUT24 059

×

SelectChessTopPlayersToolBFCL/multiple_177 (board_game.chess.get_top_players): top chess players by location/rating.1 AI query 44 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 44

Failed

IN459 280 OUT3 868

×

SelectDefenseRankingToolBFCL/multiple_173 (get_defense_ranking): NBA best-defence ranking by season.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN528 667 OUT3 752

×

SelectFindAthleteToolBFCL/multiple_172 (sports_db.find_athlete): athlete profile by name/sport.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN564 871 OUT4 422

✓

SelectFindCardToolBFCL/multiple_178 (find_card_in_deck): find a card by rank/suit in default deck.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN13 421 OUT66

×

SelectGameWinnerToolBFCL/multiple_171 (game_result.get_winner): winner of a game between two teams on a date.1 AI query 11 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 11

Failed

IN90 120 OUT946

×

SelectPokerFullHouseToolBFCL/multiple_179 (poker_probability.full_house): probability of a full house (default deck/hand).1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN78 202 OUT703

×

SelectSoccerPlayerStatsToolBFCL/multiple_170 (soccer_stat.get_player_stats): player stats by name/season.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN551 854 OUT4 881

×

SelectSportsRankingToolBFCL/multiple_174 (sports_ranking): team ranking in a league.1 AI query 43 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 43

Failed

IN440 457 OUT3 752

×

SelectTeamScheduleToolBFCL/multiple_176 (sports_team.get_schedule): team schedule for N games in a league.1 AI query 9 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 9

Failed

IN73 140 OUT704

×

SelectTopPlayerToolBFCL/multiple_175 (sports_ranking.get_top_player): top player by sport/gender.1 AI query 13 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 13

Failed

IN105 560 OUT965

Tool Calling - Tool Selection: Statistics, Finance & Health 2 passed4 failed

33.3%

IN820 541 OUT9 114

✓

SelectBmiMetersToolBFCL/multiple_117 (calculate_BMI): BMI in kg/m, not soccer / hotel_booking distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN9 371 OUT102

×

SelectGcdToolBFCL/multiple_101 (math.gcd): greatest common divisor, not get_top_cases.1 AI query 5 tool callscategory:ToolCallinglevel:Easysource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN29 602 OUT296

×

SelectHotelPricingToolBFCL/multiple_94 (hotel_room_pricing.get): hotel room pricing, not car/flight pricing.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN533 734 OUT4 831

×

SelectReturnOnEquityToolBFCL/multiple_128 (calculate_return_on_equity): ROE from income/equity/dividends, not library/lyrics/law distractors. dividends_paid is required 200000: prompt explicitly supplies "dividends paid of $200,000".1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN21 957 OUT449

✓

SelectSingleProbabilityToolBFCL/multiple_124 (probabilities.calculate_single): single-event probability, not math.power / fetch_DNA_sequence.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Passed

IN9 357 OUT152

×

SelectTwoSampleTtestToolBFCL/multiple_125 (run_two_sample_ttest): two-group t-test, not restaurant/personality distractors. equal_variance is required true: prompt says "assuming equal variance" (ground truth = [true, ""]).1 AI query 29 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 29

Failed

IN216 520 OUT3 284

Tool Calling - Tool Selection: Statistics, Finance & Services 2 passed6 failed

25.0%

IN1 420 903 OUT15 308

×

SelectBmiMetricToolBFCL/multiple_116 (calculate_bmi): BMI in kg/cm with unit option, not celebrity_net_worth.1 AI query 10 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 10

Failed

IN69 505 OUT771

×

SelectCalculateProbabilityToolBFCL/multiple_123 (calculate_probability): probability from favorable/total, not lawsuit/house-price distractors.1 AI query 4 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 4

Failed

IN29 434 OUT454

×

SelectCompoundInterestToolBFCL/multiple_129 (compound_interest): compound interest, not lawsuits_search.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN605 994 OUT5 446

×

SelectFindRestaurantsToolBFCL/multiple_115 (find_restaurants): restaurant search with count + dietary, not instrument/stock/sports.1 AI query 35 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 35

Failed

IN311 439 OUT3 503

✓

SelectLinearRegressionToolBFCL/multiple_122 (run_linear_regression): regression with predictors, not recipe/itinerary/cooking distractors. standardize is required true: prompt says "Also apply standardization" (ground truth = [true]).1 AI query 1 tool callcategory:ToolCallinglevel:Hardsource:BFCL

Passed

IN11 421 OUT126

×

SelectMoviesByActorToolBFCL/multiple_118 (imdb.find_movies_by_actor): movies by actor+year, not flight.book / lawsuits_search.1 AI query 39 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 39

Failed

IN345 561 OUT3 456

✓

SelectQuarterlyDividendToolBFCL/multiple_127 (finance.calculate_quarterly_dividend_per_share): dividend per share, not lyrics/paint/movie distractors.1 AI query 1 tool callcategory:ToolCallinglevel:Easysource:BFCL

Passed

IN11 425 OUT133

×

SelectTTestToolBFCL/multiple_126 (t_test): two-dataset t-test, not event_finder / triangle-area distractors.1 AI query 5 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN36 124 OUT1 419

Tool Calling - Tool Selection: Travel & Biology 8 failed

0.0%

IN1 773 142 OUT15 904

×

SelectBookHotelToolBFCL/multiple_190 (book_hotel): book a room by hotel/location/room_type/start/nights.1 AI query 5 tool callscategory:ToolCallinglevel:Hardsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 5

Failed

IN38 875 OUT574

×

SelectCellProteinsToolBFCL/multiple_196 (cellbio.get_proteins): proteins in a cell compartment.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN508 320 OUT4 381

×

SelectDistanceDurationToolBFCL/multiple_193 (maps.get_distance_duration): distance/duration between two places.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN24 983 OUT252

×

SelectForestGrowthToolBFCL/multiple_199 (forest_growth_forecast): forest growth forecast with human impact.1 AI query 50 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.Completed Values: response.Completed = false

Failed

IN507 145 OUT4 184

×

SelectGenotypeFrequencyToolBFCL/multiple_198 (calculate_genotype_frequency): genotype frequency from allele frequency.1 AI query 1 tool callcategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: args.genotype == "AA" Values: args.genotype = "homozygous dominant"

Failed

IN12 251 OUT139

×

SelectHeatCapacityToolBFCL/multiple_195 (calc_heat_capacity): heat capacity for a gas by temp/volume. Tightened per methodology: prompt explicitly states the gas is "air", so gas is required exactly "air" (ground truth also accepts omission, which equals the schema default).1 AI query 12 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 12

Failed

IN90 308 OUT968

×

SelectMuseumHoursToolBFCL/multiple_194 (get_museum_hours): museum opening hours on a day.1 AI query 38 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 38

Failed

IN346 652 OUT3 154

×

SelectMutationTypeToolBFCL/multiple_197 (mutation_type.find): gene mutation type by SNP id.1 AI query 29 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 29

Failed

IN244 608 OUT2 252

Tool Calling - Tool Selection: Triangle Properties 1 failed

0.0%

IN9 885 OUT509

×

SelectTrianglePropertiesToolBFCL/multiple_0 (triangle_properties.get): retrieves triangle dimensions/properties from 3 sides.1 AI query 3 tool callscategory:ToolCallinglevel:Mediumsource:BFCL

Assertion expression evaluated to false: response.ToolCalls.Count == 1 Values: response.ToolCalls.Count = 3

Failed

IN9 885 OUT509

Qwen 3.6 35B FP8 Instruct | Benchmark #17

Tool Calling Benchmark based on BFCL

Results by difficulty

Results by category