Benchmark summary
BBH-based reasoning benchmark ported to the SC# evaluation runtime. It tests multi-step reasoning across challenging tasks involving logic, deduction, symbolic manipulation, planning, and problem solving.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Reasoning - BBH boolean expressions (set 1)
32 passed |
100.0% |
IN30 784 OUT27 650 | |
| ✓ | BbhBooleanExpressionsBD1not True or ( False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT733 |
| ✓ | BbhBooleanExpressionsBD10( True and not True and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT537 |
| ✓ | BbhBooleanExpressionsBD11False and False and False or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT500 |
| ✓ | BbhBooleanExpressionsBD12False or ( False and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT574 |
| ✓ | BbhBooleanExpressionsBD13True and False or ( not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT787 |
| ✓ | BbhBooleanExpressionsBD14not not ( True ) and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 316 |
| ✓ | BbhBooleanExpressionsBD15not False or ( True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 011 |
| ✓ | BbhBooleanExpressionsBD16not ( True ) or False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT491 |
| ✓ | BbhBooleanExpressionsBD17( True and not not not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT639 |
| ✓ | BbhBooleanExpressionsBD18( False or not False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT715 |
| ✓ | BbhBooleanExpressionsBD19False and False or True and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 246 |
| ✓ | BbhBooleanExpressionsBD2not True or ( True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT627 |
| ✓ | BbhBooleanExpressionsBD20not not False or not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 133 |
| ✓ | BbhBooleanExpressionsBD21True and not True or False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 039 |
| ✓ | BbhBooleanExpressionsBD22not False or True and False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 405 |
| ✓ | BbhBooleanExpressionsBD23False and ( not False ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT433 |
| ✓ | BbhBooleanExpressionsBD24not not not not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT838 |
| ✓ | BbhBooleanExpressionsBD25( True ) and True or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT800 |
| ✓ | BbhBooleanExpressionsBD26not True or False and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT728 |
| ✓ | BbhBooleanExpressionsBD27not not not not not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT654 |
| ✓ | BbhBooleanExpressionsBD28not True or not False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT589 |
| ✓ | BbhBooleanExpressionsBD29False or True or False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT2 524 |
| ✓ | BbhBooleanExpressionsBD3False or ( False ) and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT700 |
| ✓ | BbhBooleanExpressionsBD30not ( True or True ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT758 |
| ✓ | BbhBooleanExpressionsBD31not not not not True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT805 |
| ✓ | BbhBooleanExpressionsBD32( False ) or False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT730 |
| ✓ | BbhBooleanExpressionsBD4not False or True and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 241 |
| ✓ | BbhBooleanExpressionsBD5not True or False or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT595 |
| ✓ | BbhBooleanExpressionsBD6True and True and False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT702 |
| ✓ | BbhBooleanExpressionsBD7not not True and not True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 153 |
| ✓ | BbhBooleanExpressionsBD8not not not ( True and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT454 |
| ✓ | BbhBooleanExpressionsBD9not not False and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 193 |
Reasoning - BBH boolean expressions (set 2)
31 passed1 failed |
96.9% |
IN30 784 OUT27 819 | |
| ✓ | BbhBooleanExpressionsBD33( False and True or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT877 |
| ✓ | BbhBooleanExpressionsBD34not not not ( False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT952 |
| ✓ | BbhBooleanExpressionsBD35not not True and not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 006 |
| ✓ | BbhBooleanExpressionsBD36not False or True and not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 223 |
| ✓ | BbhBooleanExpressionsBD37True or True or not False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT998 |
| ✓ | BbhBooleanExpressionsBD38( not False and False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT619 |
| ✓ | BbhBooleanExpressionsBD39( ( not True ) or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT432 |
| ✓ | BbhBooleanExpressionsBD40not not ( not False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT608 |
| ✓ | BbhBooleanExpressionsBD41not True or False or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 198 |
| ✓ | BbhBooleanExpressionsBD42False or ( not False ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT637 |
| ✓ | BbhBooleanExpressionsBD43not False or False or ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT599 |
| ✓ | BbhBooleanExpressionsBD44not not False or False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT987 |
| ✓ | BbhBooleanExpressionsBD45True or ( ( not True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT795 |
| ✓ | BbhBooleanExpressionsBD46not False and not False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT495 |
| ✓ | BbhBooleanExpressionsBD47( False ) and not False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 219 |
| ✓ | BbhBooleanExpressionsBD48( True ) and False or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT979 |
| ✓ | BbhBooleanExpressionsBD49( not False and ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT615 |
| ✓ | BbhBooleanExpressionsBD50False or not ( False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 045 |
| ✓ | BbhBooleanExpressionsBD51True and not False and not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT818 |
| ✓ | BbhBooleanExpressionsBD52not True and True and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 086 |
| ✓ | BbhBooleanExpressionsBD53not not ( False and not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT677 |
| × | BbhBooleanExpressionsBD54True or not ( ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("True")
Context: BbhBooleanExpressionsBD54: expected True, got [false]
Values:
got = "false" |
Failed | IN962 OUT948 |
| ✓ | BbhBooleanExpressionsBD55False or not True or False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT508 |
| ✓ | BbhBooleanExpressionsBD56False or not not True or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT2 283 |
| ✓ | BbhBooleanExpressionsBD57not ( True or True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT381 |
| ✓ | BbhBooleanExpressionsBD58False or not not not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 332 |
| ✓ | BbhBooleanExpressionsBD59not ( False ) or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT917 |
| ✓ | BbhBooleanExpressionsBD60True and not False or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT786 |
| ✓ | BbhBooleanExpressionsBD61True or ( not ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT527 |
| ✓ | BbhBooleanExpressionsBD62( False or True and not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT651 |
| ✓ | BbhBooleanExpressionsBD63False or not True and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT997 |
| ✓ | BbhBooleanExpressionsBD64not not not not not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT624 |
Reasoning - BBH boolean expressions (set 3)
32 passed |
100.0% |
IN30 784 OUT26 001 | |
| ✓ | BbhBooleanExpressionsBD65( not False ) and False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT395 |
| ✓ | BbhBooleanExpressionsBD66False and False or not ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT754 |
| ✓ | BbhBooleanExpressionsBD67( not False or True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT690 |
| ✓ | BbhBooleanExpressionsBD68not ( False ) and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT969 |
| ✓ | BbhBooleanExpressionsBD69False or True and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT971 |
| ✓ | BbhBooleanExpressionsBD70( True ) and True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 279 |
| ✓ | BbhBooleanExpressionsBD71( not True ) and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT334 |
| ✓ | BbhBooleanExpressionsBD72( not not True and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT440 |
| ✓ | BbhBooleanExpressionsBD73True and ( False ) or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 093 |
| ✓ | BbhBooleanExpressionsBD74not not not False or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT647 |
| ✓ | BbhBooleanExpressionsBD75not not not False and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT2 349 |
| ✓ | BbhBooleanExpressionsBD76True or ( True or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT512 |
| ✓ | BbhBooleanExpressionsBD77not False or ( ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT740 |
| ✓ | BbhBooleanExpressionsBD78not not ( False and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT528 |
| ✓ | BbhBooleanExpressionsBD79not not not not True or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT607 |
| ✓ | BbhBooleanExpressionsBD80False and not ( False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT960 |
| ✓ | BbhBooleanExpressionsBD81True and True and True or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT634 |
| ✓ | BbhBooleanExpressionsBD82False or not False or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT949 |
| ✓ | BbhBooleanExpressionsBD83not False and False and False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT620 |
| ✓ | BbhBooleanExpressionsBD84( False ) or not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT797 |
| ✓ | BbhBooleanExpressionsBD85( not False ) or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT343 |
| ✓ | BbhBooleanExpressionsBD86not not ( False or not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT676 |
| ✓ | BbhBooleanExpressionsBD87False or not False or False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 206 |
| ✓ | BbhBooleanExpressionsBD88False or not False and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 113 |
| ✓ | BbhBooleanExpressionsBD89False or ( False ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 038 |
| ✓ | BbhBooleanExpressionsBD90not True or False and True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 103 |
| ✓ | BbhBooleanExpressionsBD91True or ( True or not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT499 |
| ✓ | BbhBooleanExpressionsBD92True and not False and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT887 |
| ✓ | BbhBooleanExpressionsBD93False or not False or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT801 |
| ✓ | BbhBooleanExpressionsBD94( True or not False ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT586 |
| ✓ | BbhBooleanExpressionsBD95False or not ( True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT580 |
| ✓ | BbhBooleanExpressionsBD96not False and not True and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT901 |
Reasoning - BBH boolean expressions (set 4)
30 passed2 failed |
93.8% |
IN30 784 OUT25 232 | |
| ✓ | BbhBooleanExpressionsBD100( True or not not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT759 |
| ✓ | BbhBooleanExpressionsBD101False and ( False or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT807 |
| ✓ | BbhBooleanExpressionsBD102False or not True or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 304 |
| ✓ | BbhBooleanExpressionsBD103True and not True or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 020 |
| ✓ | BbhBooleanExpressionsBD104( not False or ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT619 |
| ✓ | BbhBooleanExpressionsBD105( ( not ( True ) ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT374 |
| × | BbhBooleanExpressionsBD106False and False and not ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("False")
Context: BbhBooleanExpressionsBD106: expected False, got [** false]
Values:
got = "** false" |
Failed | IN962 OUT435 |
| ✓ | BbhBooleanExpressionsBD107not True and not True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT966 |
| ✓ | BbhBooleanExpressionsBD108not ( True or False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT855 |
| ✓ | BbhBooleanExpressionsBD109False or ( True ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT786 |
| ✓ | BbhBooleanExpressionsBD110False or False and False or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT830 |
| × | BbhBooleanExpressionsBD111True or False or not True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("True")
Context: BbhBooleanExpressionsBD111: expected True, got [false]
Values:
got = "false" |
Failed | IN962 OUT767 |
| ✓ | BbhBooleanExpressionsBD112not True and ( True and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 081 |
| ✓ | BbhBooleanExpressionsBD113not True and True or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 202 |
| ✓ | BbhBooleanExpressionsBD114not False or ( True or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT462 |
| ✓ | BbhBooleanExpressionsBD115not ( False or ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT451 |
| ✓ | BbhBooleanExpressionsBD116not ( True or True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT509 |
| ✓ | BbhBooleanExpressionsBD117False or not False or True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 195 |
| ✓ | BbhBooleanExpressionsBD118False or False and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 174 |
| ✓ | BbhBooleanExpressionsBD119not ( True or False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT427 |
| ✓ | BbhBooleanExpressionsBD120( False ) or not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 388 |
| ✓ | BbhBooleanExpressionsBD121True and True and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT930 |
| ✓ | BbhBooleanExpressionsBD122( True ) or True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 045 |
| ✓ | BbhBooleanExpressionsBD123False and False and not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT847 |
| ✓ | BbhBooleanExpressionsBD124( True ) and not True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT596 |
| ✓ | BbhBooleanExpressionsBD125not not not True and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT586 |
| ✓ | BbhBooleanExpressionsBD126False and False and not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 252 |
| ✓ | BbhBooleanExpressionsBD127False and False or False or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT663 |
| ✓ | BbhBooleanExpressionsBD128False or ( not ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT419 |
| ✓ | BbhBooleanExpressionsBD97False and ( True and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT339 |
| ✓ | BbhBooleanExpressionsBD98not False or ( False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT529 |
| ✓ | BbhBooleanExpressionsBD99False and not ( False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT615 |
Reasoning - BBH boolean expressions (set 5)
30 passed |
100.0% |
IN28 860 OUT26 955 | |
| ✓ | BbhBooleanExpressionsBD129not True and True and not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 148 |
| ✓ | BbhBooleanExpressionsBD130( False ) and ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT379 |
| ✓ | BbhBooleanExpressionsBD131( not True ) and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 035 |
| ✓ | BbhBooleanExpressionsBD132False and True and ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT450 |
| ✓ | BbhBooleanExpressionsBD133( ( True ) ) or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT678 |
| ✓ | BbhBooleanExpressionsBD134not True and False or False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 417 |
| ✓ | BbhBooleanExpressionsBD135True and not True and True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 607 |
| ✓ | BbhBooleanExpressionsBD136( True or True or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT573 |
| ✓ | BbhBooleanExpressionsBD137True or not False and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT835 |
| ✓ | BbhBooleanExpressionsBD138( not ( False ) or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT370 |
| ✓ | BbhBooleanExpressionsBD139True and not ( True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT450 |
| ✓ | BbhBooleanExpressionsBD140not ( False and True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT590 |
| ✓ | BbhBooleanExpressionsBD141not ( True and True ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT341 |
| ✓ | BbhBooleanExpressionsBD142False or not True and not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT776 |
| ✓ | BbhBooleanExpressionsBD143not True and False or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 087 |
| ✓ | BbhBooleanExpressionsBD144not True or True and False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT2 332 |
| ✓ | BbhBooleanExpressionsBD145( not ( False ) and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT431 |
| ✓ | BbhBooleanExpressionsBD146False or ( True and not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT610 |
| ✓ | BbhBooleanExpressionsBD147not False and not True or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 531 |
| ✓ | BbhBooleanExpressionsBD148( False or not not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 039 |
| ✓ | BbhBooleanExpressionsBD149( not False ) or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT915 |
| ✓ | BbhBooleanExpressionsBD150( not not True ) or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT915 |
| ✓ | BbhBooleanExpressionsBD151True or not ( ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT667 |
| ✓ | BbhBooleanExpressionsBD152( False ) and False or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 354 |
| ✓ | BbhBooleanExpressionsBD153not False and True or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 062 |
| ✓ | BbhBooleanExpressionsBD154True or not not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT757 |
| ✓ | BbhBooleanExpressionsBD155False and not True and False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT981 |
| ✓ | BbhBooleanExpressionsBD156not not True and True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 224 |
| ✓ | BbhBooleanExpressionsBD157not False or not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT700 |
| ✓ | BbhBooleanExpressionsBD158not not ( True ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT701 |
Reasoning - BBH boolean expressions (set 6)
32 passed |
100.0% |
IN30 784 OUT28 611 | |
| ✓ | BbhBooleanExpressionsBe166( True and not True and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT631 |
| ✓ | BbhBooleanExpressionsBe167True or False or not True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 404 |
| ✓ | BbhBooleanExpressionsBe168not True or True and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 524 |
| ✓ | BbhBooleanExpressionsBe169True and not not not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT890 |
| ✓ | BbhBooleanExpressionsBe170not True and ( False ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT846 |
| ✓ | BbhBooleanExpressionsBe171False and False or not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT858 |
| ✓ | BbhBooleanExpressionsBe172True or False and not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 695 |
| ✓ | BbhBooleanExpressionsBe173not ( False or False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT379 |
| ✓ | BbhBooleanExpressionsBe174not False or True or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 020 |
| ✓ | BbhBooleanExpressionsBe175not False and False or False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT660 |
| ✓ | BbhBooleanExpressionsBe176( not True ) and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT566 |
| ✓ | BbhBooleanExpressionsBe177True and not not False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT709 |
| ✓ | BbhBooleanExpressionsBe178not ( not not False ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 112 |
| ✓ | BbhBooleanExpressionsBe179not ( not False ) and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT459 |
| ✓ | BbhBooleanExpressionsBe180False or True and not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT844 |
| ✓ | BbhBooleanExpressionsBe181( not not not True and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT886 |
| ✓ | BbhBooleanExpressionsBe182False and not ( True ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT784 |
| ✓ | BbhBooleanExpressionsBe183( not True and False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 594 |
| ✓ | BbhBooleanExpressionsBe184not ( not not True ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT852 |
| ✓ | BbhBooleanExpressionsBe185( not False ) or ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT317 |
| ✓ | BbhBooleanExpressionsBe186not not not False and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT872 |
| ✓ | BbhBooleanExpressionsBe187True or not True or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 036 |
| ✓ | BbhBooleanExpressionsBe188not False or not not False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 317 |
| ✓ | BbhBooleanExpressionsBe189not False and ( False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT785 |
| ✓ | BbhBooleanExpressionsBe190False and True or ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 223 |
| ✓ | BbhBooleanExpressionsBe191False and not False and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT544 |
| ✓ | BbhBooleanExpressionsBe192not ( ( False and True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT468 |
| ✓ | BbhBooleanExpressionsBe193False or ( not False ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT926 |
| ✓ | BbhBooleanExpressionsBe194False and not False or False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT725 |
| ✓ | BbhBooleanExpressionsBe195not False and not not True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT597 |
| ✓ | BbhBooleanExpressionsBe196True and True or False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT900 |
| ✓ | BbhBooleanExpressionsBe197not True or False and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 188 |
Reasoning - BBH boolean expressions (set 7)
32 passed |
100.0% |
IN30 784 OUT27 332 | |
| ✓ | BbhBooleanExpressionsBe198True or False or not True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 958 |
| ✓ | BbhBooleanExpressionsBe199False or False or not True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT683 |
| ✓ | BbhBooleanExpressionsBe200False or False or True and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT591 |
| ✓ | BbhBooleanExpressionsBe201( not ( True or False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT477 |
| ✓ | BbhBooleanExpressionsBe202not False and False or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT737 |
| ✓ | BbhBooleanExpressionsBe203True and False or ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT899 |
| ✓ | BbhBooleanExpressionsBe204not False or not not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT611 |
| ✓ | BbhBooleanExpressionsBe205True and ( not ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT305 |
| ✓ | BbhBooleanExpressionsBe206True and False and not True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 312 |
| ✓ | BbhBooleanExpressionsBe207( ( not False ) ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT455 |
| ✓ | BbhBooleanExpressionsBe208not ( True ) and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT686 |
| ✓ | BbhBooleanExpressionsBe209True or not False and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 283 |
| ✓ | BbhBooleanExpressionsBe210True and not ( False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT527 |
| ✓ | BbhBooleanExpressionsBe211not True and False or False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT844 |
| ✓ | BbhBooleanExpressionsBe212not True or not False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 247 |
| ✓ | BbhBooleanExpressionsBe213not True or False or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT636 |
| ✓ | BbhBooleanExpressionsBe214not True and True and False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT916 |
| ✓ | BbhBooleanExpressionsBe215False and False or not True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT627 |
| ✓ | BbhBooleanExpressionsBe216( not False or False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT890 |
| ✓ | BbhBooleanExpressionsBe217( True or False ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT974 |
| ✓ | BbhBooleanExpressionsBe218not not not ( False ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT625 |
| ✓ | BbhBooleanExpressionsBe219not False and False and False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 149 |
| ✓ | BbhBooleanExpressionsBe220False and not True and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT577 |
| ✓ | BbhBooleanExpressionsBe221( False or not False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT603 |
| ✓ | BbhBooleanExpressionsBe222False or ( False or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT556 |
| ✓ | BbhBooleanExpressionsBe223not False or True or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT2 047 |
| ✓ | BbhBooleanExpressionsBe224True or ( not False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT669 |
| ✓ | BbhBooleanExpressionsBe225not False and False or True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 101 |
| ✓ | BbhBooleanExpressionsBe226False and not ( not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT883 |
| ✓ | BbhBooleanExpressionsBe227not not ( not True ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT793 |
| ✓ | BbhBooleanExpressionsBe228False and not True and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT824 |
| ✓ | BbhBooleanExpressionsBe229True or False and not ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT847 |
Reasoning - BBH boolean expressions (set 8)
28 passed4 failed |
87.5% |
IN33 111 OUT41 598 | |
| ✓ | BbhBooleanExpressionsBe230not ( False ) and ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT805 |
| ✓ | BbhBooleanExpressionsBe231False and not ( ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT624 |
| ✓ | BbhBooleanExpressionsBe232not False or ( not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT647 |
| ✓ | BbhBooleanExpressionsBe233( ( not False ) and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT398 |
| ✓ | BbhBooleanExpressionsBe234True or False and not False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 808 |
| ✓ | BbhBooleanExpressionsBe235( ( not True and True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT956 |
| ✓ | BbhBooleanExpressionsBe236not ( False ) or not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT824 |
| ✓ | BbhBooleanExpressionsBe237not ( False or False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 623 |
| ✓ | BbhBooleanExpressionsBe238False and not ( not not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT955 |
| ✓ | BbhBooleanExpressionsBe239False and False and not ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT815 |
| ✓ | BbhBooleanExpressionsBe240False or ( not False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT887 |
| ✓ | BbhBooleanExpressionsBe241False or ( not not not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT695 |
| ✓ | BbhBooleanExpressionsBe242( not False ) or not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT757 |
| ✓ | BbhBooleanExpressionsBe243not False and False and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 704 |
| ✓ | BbhBooleanExpressionsBe244True or not False and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT658 |
| ✓ | BbhBooleanExpressionsBe245( not ( False ) ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT434 |
| ✓ | BbhBooleanExpressionsBe246True and not True or False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT974 |
| ✓ | BbhBooleanExpressionsBe247not ( not not not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT525 |
| ✓ | BbhBooleanExpressionsBe248True and ( not True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT652 |
| ✓ | BbhBooleanExpressionsBe249( not not not False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT623 |
| × | BbhCausalJudgementBe10judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("Yes")
Context: BbhCausalJudgementBe10: expected Yes, got [No] |
Failed | IN1 111 OUT1 920 |
| × | BbhCausalJudgementBe11judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("No")
Context: BbhCausalJudgementBe11: expected No, got [Yes] |
Failed | IN1 127 OUT1 901 |
| ✓ | BbhCausalJudgementBe12judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 940 |
| ✓ | BbhCausalJudgementBe13judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 166 OUT1 963 |
| × | BbhCausalJudgementBe14judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("Yes")
Context: BbhCausalJudgementBe14: expected Yes, got [No] |
Failed | IN1 145 OUT2 969 |
| ✓ | BbhCausalJudgementBe15judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 180 OUT1 186 |
| × | BbhCausalJudgementBe16judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("Yes")
Context: BbhCausalJudgementBe16: expected Yes, got [No] |
Failed | IN1 168 OUT2 620 |
| ✓ | BbhCausalJudgementBe17judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT2 701 |
| ✓ | BbhCausalJudgementBe18judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 351 OUT2 508 |
| ✓ | BbhCausalJudgementBe19judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT939 |
| ✓ | BbhCausalJudgementBe8judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 439 |
| ✓ | BbhCausalJudgementBe9judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT2 148 |
Reasoning - BBH Dyck languages - close the brackets (set 1)
26 passed3 failed1 broken |
86.7% |
IN29 711 OUT124 653 | |
| ✓ | BbhDyckLanguagesBi13correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT948 |
| × | BbhDyckLanguagesBi14correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> )")
Context: BbhDyckLanguagesBi14: expected > ), got [)>] |
Failed | IN1 007 OUT14 083 |
| ✓ | BbhDyckLanguagesBi15correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT1 461 |
| ✓ | BbhDyckLanguagesBi16correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT1 872 |
| ✓ | BbhDyckLanguagesBi17correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 385 |
| ✓ | BbhDyckLanguagesBi18correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT1 247 |
| ✓ | BbhDyckLanguagesBi19correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 153 |
| ! | BbhDyckLanguagesBi20correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 062 OUT17 777 |
| ✓ | BbhDyckLanguagesBi21correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 368 |
| × | BbhDyckLanguagesBi22correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} ] >")
Context: BbhDyckLanguagesBi22: expected } ] >, got [}>]] |
Failed | IN1 024 OUT7 242 |
| ✓ | BbhDyckLanguagesBi23correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT4 677 |
| ✓ | BbhDyckLanguagesBi24correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT5 764 |
| ✓ | BbhDyckLanguagesBi25correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT2 266 |
| ✓ | BbhDyckLanguagesBi26correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT1 141 |
| ✓ | BbhDyckLanguagesBi27correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT3 737 |
| × | BbhDyckLanguagesBi28correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") )")
Context: BbhDyckLanguagesBi28: expected ) ), got [( ( { } ) )] |
Failed | IN975 OUT970 |
| ✓ | BbhDyckLanguagesBi29correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT3 794 |
| ✓ | BbhDyckLanguagesBi30correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT2 783 |
| ✓ | BbhDyckLanguagesBi31correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 011 OUT6 431 |
| ✓ | BbhDyckLanguagesBi32correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 959 |
| ✓ | BbhDyckLanguagesBi33correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 020 OUT7 153 |
| ✓ | BbhDyckLanguagesBi34correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT953 |
| ✓ | BbhDyckLanguagesBi35correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 033 |
| ✓ | BbhDyckLanguagesBi36correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT3 126 |
| ✓ | BbhDyckLanguagesBi37correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT5 192 |
| ✓ | BbhDyckLanguagesBi38correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 641 |
| ✓ | BbhDyckLanguagesBi39correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 054 OUT14 889 |
| ✓ | BbhDyckLanguagesBi40correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 377 |
| ✓ | BbhDyckLanguagesBi41correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT2 100 |
| ✓ | BbhDyckLanguagesBi42correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT5 131 |
Reasoning - BBH Dyck languages - close the brackets (set 2)
25 passed5 failed2 broken |
78.1% |
IN31 926 OUT151 632 | |
| ✓ | BbhDyckLanguagesBj51correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT2 887 |
| ✓ | BbhDyckLanguagesBj52correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 719 |
| ✓ | BbhDyckLanguagesBj53correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT2 012 |
| ! | BbhDyckLanguagesBj54correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 064 OUT16 789 |
| ✓ | BbhDyckLanguagesBj55correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT2 249 |
| ✓ | BbhDyckLanguagesBj56correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT3 171 |
| × | BbhDyckLanguagesBj57correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("}")
Context: BbhDyckLanguagesBj57: expected }, got [< > {}] |
Failed | IN974 OUT4 338 |
| ✓ | BbhDyckLanguagesBj58correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 928 |
| ✓ | BbhDyckLanguagesBj59correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 012 |
| ✓ | BbhDyckLanguagesBj60correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT3 159 |
| ✓ | BbhDyckLanguagesBj61correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 008 OUT3 293 |
| ✓ | BbhDyckLanguagesBj62correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT3 477 |
| ✓ | BbhDyckLanguagesBj63correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT2 421 |
| ✓ | BbhDyckLanguagesBj64correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT1 533 |
| ✓ | BbhDyckLanguagesBj65correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT1 511 |
| × | BbhDyckLanguagesBj66correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> } ]")
Context: BbhDyckLanguagesBj66: expected > } ], got [> ]] |
Failed | IN1 026 OUT11 666 |
| ✓ | BbhDyckLanguagesBj67correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT2 489 |
| ✓ | BbhDyckLanguagesBj68correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT974 |
| ✓ | BbhDyckLanguagesBj69correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT3 697 |
| ✓ | BbhDyckLanguagesBj70correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 002 OUT5 065 |
| × | BbhDyckLanguagesBj71correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") )")
Context: BbhDyckLanguagesBj71: expected ) ), got [}]] }>))] |
Failed | IN1 027 OUT12 285 |
| ✓ | BbhDyckLanguagesBj72correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT3 611 |
| × | BbhDyckLanguagesBj73correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") )")
Context: BbhDyckLanguagesBj73: expected ) ), got [} } } } ) )] |
Failed | IN1 035 OUT14 962 |
| ! | BbhDyckLanguagesBj74correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 070 OUT13 569 |
| ✓ | BbhDyckLanguagesBj75correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 028 OUT8 829 |
| ✓ | BbhDyckLanguagesBj76correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT2 086 |
| ✓ | BbhDyckLanguagesBj77correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT4 014 |
| × | BbhDyckLanguagesBj78correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} ] >")
Context: BbhDyckLanguagesBj78: expected } ] >, got [] >] |
Failed | IN1 020 OUT4 856 |
| ✓ | BbhDyckLanguagesBj79correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 006 OUT4 190 |
| ✓ | BbhDyckLanguagesBj80correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT2 399 |
| ✓ | BbhDyckLanguagesBj81correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT4 029 |
| ✓ | BbhDyckLanguagesBj82correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 412 |
Reasoning - BBH Dyck languages - close the brackets (set 3)
22 passed9 failed1 broken |
68.8% |
IN31 902 OUT202 667 | |
| ✓ | BbhDyckLanguagesBj100correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT4 237 |
| ✓ | BbhDyckLanguagesBj101correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT3 075 |
| ✓ | BbhDyckLanguagesBj102correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 045 OUT10 544 |
| ✓ | BbhDyckLanguagesBj103correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 128 |
| ✓ | BbhDyckLanguagesBj104correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 750 |
| ✓ | BbhDyckLanguagesBj105correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 041 |
| × | BbhDyckLanguagesBj106correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("]")
Context: BbhDyckLanguagesBj106: expected ], got [] |
Failed | IN1 008 OUT6 372 |
| ✓ | BbhDyckLanguagesBj107correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 319 |
| ✓ | BbhDyckLanguagesBj108correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT2 873 |
| ! | BbhDyckLanguagesBj109correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 064 OUT62 144 |
| ✓ | BbhDyckLanguagesBj110correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 730 |
| ✓ | BbhDyckLanguagesBj111correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 010 OUT5 380 |
| ✓ | BbhDyckLanguagesBj112correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 063 OUT7 515 |
| ✓ | BbhDyckLanguagesBj113correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT804 |
| ✓ | BbhDyckLanguagesBj114correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT3 108 |
| ✓ | BbhDyckLanguagesBj83correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT4 356 |
| × | BbhDyckLanguagesBj84correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} >")
Context: BbhDyckLanguagesBj84: expected } >, got [}>}] |
Failed | IN981 OUT3 427 |
| ✓ | BbhDyckLanguagesBj85correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT2 000 |
| ✓ | BbhDyckLanguagesBj86correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT900 |
| × | BbhDyckLanguagesBj87correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] ]")
Context: BbhDyckLanguagesBj87: expected ] ], got []])]]] |
Failed | IN1 029 OUT10 161 |
| ✓ | BbhDyckLanguagesBj88correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 057 OUT23 163 |
| ✓ | BbhDyckLanguagesBj89correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT1 503 |
| ✓ | BbhDyckLanguagesBj90correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT8 113 |
| ✓ | BbhDyckLanguagesBj91correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT2 577 |
| ✓ | BbhDyckLanguagesBj92correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 721 |
| × | BbhDyckLanguagesBj93correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] ]")
Context: BbhDyckLanguagesBj93: expected ] ], got [[ < > [ [ < [ ] > ] ]]] |
Failed | IN981 OUT2 019 |
| ✓ | BbhDyckLanguagesBj94correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 015 OUT9 417 |
| × | BbhDyckLanguagesBj95correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") )")
Context: BbhDyckLanguagesBj95: expected ) ), got [> ] )] |
Failed | IN1 057 OUT7 918 |
| × | BbhDyckLanguagesBj96correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(">")
Context: BbhDyckLanguagesBj96: expected >, got [< < { } > >] |
Failed | IN976 OUT1 312 |
| × | BbhDyckLanguagesBj97correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBj97: expected ), got [( < > )] |
Failed | IN974 OUT4 247 |
| × | BbhDyckLanguagesBj98correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} )")
Context: BbhDyckLanguagesBj98: expected } ), got [( { ( ) } )] |
Failed | IN975 OUT1 123 |
| × | BbhDyckLanguagesBj99correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") )")
Context: BbhDyckLanguagesBj99: expected ) ), got []))))] |
Failed | IN993 OUT5 690 |
Reasoning - BBH Dyck languages - close the brackets (set 4)
23 passed8 failed1 broken |
71.9% |
IN31 845 OUT161 973 | |
| ✓ | BbhDyckLanguagesBj115correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 699 |
| ✓ | BbhDyckLanguagesBj116correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 003 OUT3 545 |
| ✓ | BbhDyckLanguagesBj117correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 005 OUT4 803 |
| ✓ | BbhDyckLanguagesBj118correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT4 694 |
| ✓ | BbhDyckLanguagesBj119correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT2 207 |
| ! | BbhDyckLanguagesBj120correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 017 OUT16 090 |
| ✓ | BbhDyckLanguagesBj121correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT3 195 |
| ✓ | BbhDyckLanguagesBj122correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 040 OUT11 761 |
| ✓ | BbhDyckLanguagesBj123correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT1 003 |
| ✓ | BbhDyckLanguagesBj124correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 002 OUT4 777 |
| ✓ | BbhDyckLanguagesBj125correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT5 164 |
| ✓ | BbhDyckLanguagesBj126correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT5 572 |
| ✓ | BbhDyckLanguagesBj127correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 015 OUT6 432 |
| × | BbhDyckLanguagesBj128correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(">")
Context: BbhDyckLanguagesBj128: expected >, got [< ( ) >] |
Failed | IN974 OUT1 075 |
| ✓ | BbhDyckLanguagesBj129correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 198 |
| ✓ | BbhDyckLanguagesBj130correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT4 287 |
| ✓ | BbhDyckLanguagesBj131correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 007 OUT3 581 |
| ✓ | BbhDyckLanguagesBj132correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT2 618 |
| × | BbhDyckLanguagesBj133correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} >")
Context: BbhDyckLanguagesBj133: expected } >, got [}>}] |
Failed | IN985 OUT2 117 |
| × | BbhDyckLanguagesBj134correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] ]")
Context: BbhDyckLanguagesBj134: expected ] ], got [[ < [ ] > [ ] ]] |
Failed | IN977 OUT1 363 |
| ✓ | BbhDyckLanguagesBj135correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 022 OUT13 553 |
| ✓ | BbhDyckLanguagesBj136correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT1 728 |
| ✓ | BbhDyckLanguagesBj137correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT2 799 |
| ✓ | BbhDyckLanguagesBj138correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT3 619 |
| × | BbhDyckLanguagesBj139correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBj139: expected ), got [))] |
Failed | IN1 048 OUT22 629 |
| × | BbhDyckLanguagesBj140correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ) )")
Context: BbhDyckLanguagesBj140: expected > ) ), got [>)] |
Failed | IN1 008 OUT5 356 |
| × | BbhDyckLanguagesBj141correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") )")
Context: BbhDyckLanguagesBj141: expected ) ), got [( ) ( () ))] |
Failed | IN975 OUT2 735 |
| ✓ | BbhDyckLanguagesBj142correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT2 903 |
| ✓ | BbhDyckLanguagesBj143correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT2 489 |
| × | BbhDyckLanguagesBj144correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ) }")
Context: BbhDyckLanguagesBj144: expected ) ) }, got [)}] |
Failed | IN984 OUT3 509 |
| × | BbhDyckLanguagesBj145correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") > ]")
Context: BbhDyckLanguagesBj145: expected ) > ], got [> ) >] |
Failed | IN1 046 OUT9 880 |
| ✓ | BbhDyckLanguagesBj146correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT3 592 |
Reasoning - BBH Dyck languages - close the brackets (set 5)
27 passed4 failed1 broken |
84.4% |
IN31 748 OUT131 176 | |
| × | BbhDyckLanguagesBj147correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(">")
Context: BbhDyckLanguagesBj147: expected >, got [< < > >] |
Failed | IN974 OUT1 161 |
| ✓ | BbhDyckLanguagesBj148correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 016 OUT5 553 |
| ✓ | BbhDyckLanguagesBj149correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT4 136 |
| ✓ | BbhDyckLanguagesBj150correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT5 198 |
| ✓ | BbhDyckLanguagesBj151correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 230 |
| ✓ | BbhDyckLanguagesBj152correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 351 |
| × | BbhDyckLanguagesBj153correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] >")
Context: BbhDyckLanguagesBj153: expected ] >, got [] |
Failed | IN1 007 OUT9 835 |
| ✓ | BbhDyckLanguagesBj154correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 006 OUT4 434 |
| ✓ | BbhDyckLanguagesBj155correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 009 OUT5 079 |
| ✓ | BbhDyckLanguagesBj156correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 559 |
| ✓ | BbhDyckLanguagesBj157correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT4 349 |
| ✓ | BbhDyckLanguagesBj158correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT1 601 |
| ✓ | BbhDyckLanguagesBj159correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT2 698 |
| ✓ | BbhDyckLanguagesBj160correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 018 OUT7 867 |
| ✓ | BbhDyckLanguagesBj161correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT2 144 |
| ✓ | BbhDyckLanguagesBj162correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT1 569 |
| ✓ | BbhDyckLanguagesBj163correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 230 |
| × | BbhDyckLanguagesBj164correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> }")
Context: BbhDyckLanguagesBj164: expected > }, got [( ) { <> }] |
Failed | IN975 OUT4 256 |
| ✓ | BbhDyckLanguagesBj165correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT3 681 |
| ✓ | BbhDyckLanguagesBj166correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT2 220 |
| ✓ | BbhDyckLanguagesBj167correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 006 OUT3 525 |
| × | BbhDyckLanguagesBj168correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} }")
Context: BbhDyckLanguagesBj168: expected } }, got [{ { } }] |
Failed | IN973 OUT2 047 |
| ✓ | BbhDyckLanguagesBj169correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT4 629 |
| ✓ | BbhDyckLanguagesBj170correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 021 OUT5 622 |
| ✓ | BbhDyckLanguagesBj171correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT9 839 |
| ✓ | BbhDyckLanguagesBj172correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 507 |
| ✓ | BbhDyckLanguagesBj173correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT5 872 |
| ! | BbhDyckLanguagesBj174correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 062 OUT15 528 |
| ✓ | BbhDyckLanguagesBj175correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 002 OUT3 569 |
| ✓ | BbhDyckLanguagesBj176correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT3 667 |
| ✓ | BbhDyckLanguagesBj177correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT2 157 |
| ✓ | BbhDyckLanguagesBj178correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT2 063 |
Reasoning - BBH Dyck languages - close the brackets (set 6)
20 passed9 failed1 broken |
66.7% |
IN29 880 OUT184 409 | |
| ✓ | BbhDyckLanguagesBj179correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 640 |
| ✓ | BbhDyckLanguagesBj180correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT3 312 |
| ✓ | BbhDyckLanguagesBj181correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 395 |
| ✓ | BbhDyckLanguagesBj182correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT4 832 |
| ✓ | BbhDyckLanguagesBj183correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT4 131 |
| ✓ | BbhDyckLanguagesBj184correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 066 OUT18 063 |
| × | BbhDyckLanguagesBj185correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] }")
Context: BbhDyckLanguagesBj185: expected ] }, got [{ [ { { { } } } ] }] |
Failed | IN979 OUT3 353 |
| ! | BbhDyckLanguagesBj186correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 034 OUT21 628 |
| ✓ | BbhDyckLanguagesBj187correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT871 |
| ✓ | BbhDyckLanguagesBj188correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT1 133 |
| × | BbhDyckLanguagesBj189correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ]")
Context: BbhDyckLanguagesBj189: expected ) ], got [)]]] |
Failed | IN1 049 OUT29 312 |
| ✓ | BbhDyckLanguagesBj190correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT2 278 |
| ✓ | BbhDyckLanguagesBj191correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 585 |
| ✓ | BbhDyckLanguagesBj192correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 438 |
| × | BbhDyckLanguagesBj193correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] )")
Context: BbhDyckLanguagesBj193: expected ] ), got [( [])] |
Failed | IN973 OUT836 |
| ✓ | BbhDyckLanguagesBj194correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 002 OUT6 794 |
| × | BbhDyckLanguagesBj195correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] >")
Context: BbhDyckLanguagesBj195: expected ] >, got [] > (] |
Failed | IN1 035 OUT12 222 |
| ✓ | BbhDyckLanguagesBj196correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT1 835 |
| × | BbhDyckLanguagesBj197correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] ) ]")
Context: BbhDyckLanguagesBj197: expected ] ) ], got [[ ( [ ] ) ]] |
Failed | IN974 OUT1 023 |
| × | BbhDyckLanguagesBj198correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(">")
Context: BbhDyckLanguagesBj198: expected >, got [] |
Failed | IN1 044 OUT12 651 |
| ✓ | BbhDyckLanguagesBj199correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 019 OUT4 231 |
| × | BbhDyckLanguagesBj200correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBj200: expected ), got [>]])] |
Failed | IN1 064 OUT28 987 |
| ✓ | BbhDyckLanguagesBj201correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT2 276 |
| ✓ | BbhDyckLanguagesBj202correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 054 |
| ✓ | BbhDyckLanguagesBj203correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 430 |
| × | BbhDyckLanguagesBj204correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] ) ]")
Context: BbhDyckLanguagesBj204: expected ] ) ], got [( ) [ ( [ < { { ( { } ) } } > ] ) ]] |
Failed | IN986 OUT2 968 |
| ✓ | BbhDyckLanguagesBj205correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT2 899 |
| ✓ | BbhDyckLanguagesBj206correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT4 465 |
| × | BbhDyckLanguagesBj207correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] > }")
Context: BbhDyckLanguagesBj207: expected ] > }, got []}>] |
Failed | IN990 OUT4 387 |
| ✓ | BbhDyckLanguagesBj208correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 380 |
Reasoning - BBH Dyck languages - close the brackets (set 7)
18 passed12 failed2 broken |
56.3% |
IN31 962 OUT172 833 | |
| × | BbhDyckLanguagesBk209correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} } )")
Context: BbhDyckLanguagesBk209: expected } } ), got [}}] |
Failed | IN1 026 OUT17 221 |
| × | BbhDyckLanguagesBk210correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] }")
Context: BbhDyckLanguagesBk210: expected ] }, got [[ < [ ] > ] { [ { } ] }] |
Failed | IN983 OUT2 257 |
| × | BbhDyckLanguagesBk211correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] } )")
Context: BbhDyckLanguagesBk211: expected ] } ), got [] }] |
Failed | IN976 OUT959 |
| × | BbhDyckLanguagesBk212correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} )")
Context: BbhDyckLanguagesBk212: expected } ), got [}] |
Failed | IN1 067 OUT14 333 |
| × | BbhDyckLanguagesBk213correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") >")
Context: BbhDyckLanguagesBk213: expected ) >, got [>] |
Failed | IN987 OUT5 583 |
| ✓ | BbhDyckLanguagesBk214correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 800 |
| × | BbhDyckLanguagesBk215correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") > }")
Context: BbhDyckLanguagesBk215: expected ) > }, got [}>] |
Failed | IN1 028 OUT4 264 |
| ✓ | BbhDyckLanguagesBk216correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT3 618 |
| ✓ | BbhDyckLanguagesBk217correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT815 |
| × | BbhDyckLanguagesBk218correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBk218: expected ), got [] |
Failed | IN974 OUT1 186 |
| ✓ | BbhDyckLanguagesBk219correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT1 585 |
| ✓ | BbhDyckLanguagesBk220correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT1 632 |
| × | BbhDyckLanguagesBk221correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ] ]")
Context: BbhDyckLanguagesBk221: expected ) ] ], got []]] |
Failed | IN1 002 OUT4 579 |
| ✓ | BbhDyckLanguagesBk222correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT2 974 |
| × | BbhDyckLanguagesBk223correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> > }")
Context: BbhDyckLanguagesBk223: expected > > }, got [{ < < > > }] |
Failed | IN976 OUT1 288 |
| ✓ | BbhDyckLanguagesBk224correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT1 527 |
| ! | BbhDyckLanguagesBk225correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 064 OUT10 945 |
| ✓ | BbhDyckLanguagesBk226correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 007 OUT4 895 |
| × | BbhDyckLanguagesBk227correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} } )")
Context: BbhDyckLanguagesBk227: expected } } ), got [} } }] |
Failed | IN1 068 OUT17 536 |
| × | BbhDyckLanguagesBk228correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") >")
Context: BbhDyckLanguagesBk228: expected ) >, got [>] |
Failed | IN973 OUT1 763 |
| ✓ | BbhDyckLanguagesBk229correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT2 879 |
| ✓ | BbhDyckLanguagesBk230correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT4 857 |
| ✓ | BbhDyckLanguagesBk231correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT3 193 |
| ✓ | BbhDyckLanguagesBk232correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT3 297 |
| ✓ | BbhDyckLanguagesBk233correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT5 723 |
| ✓ | BbhDyckLanguagesBk234correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 242 |
| ✓ | BbhDyckLanguagesBk235correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 032 OUT12 291 |
| × | BbhDyckLanguagesBk236correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") } >")
Context: BbhDyckLanguagesBk236: expected ) } >, got [}>] |
Failed | IN988 OUT3 268 |
| ! | BbhDyckLanguagesBk237correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 068 OUT23 234 |
| ✓ | BbhDyckLanguagesBk238correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 556 |
| ✓ | BbhDyckLanguagesBk239correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT2 817 |
| ✓ | BbhDyckLanguagesBk240correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT7 716 |
Reasoning - BBH formal fallacies (set 1)
26 passed5 failed1 broken |
81.3% |
IN33 742 OUT138 854 | |
| × | BbhDyckLanguagesBk241correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBk241: expected ), got [] |
Failed | IN980 OUT1 338 |
| ! | BbhDyckLanguagesBk242correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN981 OUT62 144 |
| × | BbhDyckLanguagesBk243correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} )")
Context: BbhDyckLanguagesBk243: expected } ), got [}] |
Failed | IN997 OUT2 968 |
| × | BbhDyckLanguagesBk244correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ) }")
Context: BbhDyckLanguagesBk244: expected > ) }, got [{ <> }] |
Failed | IN974 OUT4 058 |
| × | BbhDyckLanguagesBk245correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} ) }")
Context: BbhDyckLanguagesBk245: expected } ) }, got [} }] |
Failed | IN996 OUT3 070 |
| ✓ | BbhDyckLanguagesBk246correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 017 OUT7 319 |
| × | BbhDyckLanguagesBk247correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ) )")
Context: BbhDyckLanguagesBk247: expected > ) ), got [>] |
Failed | IN990 OUT3 153 |
| ✓ | BbhDyckLanguagesBk248correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT3 095 |
| ✓ | BbhDyckLanguagesBk249correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT800 |
| ✓ | BbhFormalFallaciesBk10decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 299 |
| ✓ | BbhFormalFallaciesBk11decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 649 |
| ✓ | BbhFormalFallaciesBk12decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT1 468 |
| ✓ | BbhFormalFallaciesBk13decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 845 |
| ✓ | BbhFormalFallaciesBk14decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT3 488 |
| ✓ | BbhFormalFallaciesBk15decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT2 674 |
| ✓ | BbhFormalFallaciesBk16decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 969 |
| ✓ | BbhFormalFallaciesBk17decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 172 |
| ✓ | BbhFormalFallaciesBk18decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT3 118 |
| ✓ | BbhFormalFallaciesBk19decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT1 703 |
| ✓ | BbhFormalFallaciesBk20decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT3 443 |
| ✓ | BbhFormalFallaciesBk21decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT1 766 |
| ✓ | BbhFormalFallaciesBk22decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT2 000 |
| ✓ | BbhFormalFallaciesBk23decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT2 729 |
| ✓ | BbhFormalFallaciesBk24decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT3 073 |
| ✓ | BbhFormalFallaciesBk25decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 577 |
| ✓ | BbhFormalFallaciesBk26decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT1 413 |
| ✓ | BbhFormalFallaciesBk27decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT2 043 |
| ✓ | BbhFormalFallaciesBk28decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT2 561 |
| ✓ | BbhFormalFallaciesBk29decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT2 072 |
| ✓ | BbhFormalFallaciesBk30decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT5 199 |
| ✓ | BbhFormalFallaciesBk8decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT1 210 |
| ✓ | BbhFormalFallaciesBk9decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 438 |
Reasoning - BBH formal fallacies (set 2)
32 passed |
100.0% |
IN34 351 OUT67 648 | |
| ✓ | BbhFormalFallaciesBk31decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT1 755 |
| ✓ | BbhFormalFallaciesBk32decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT2 329 |
| ✓ | BbhFormalFallaciesBk33decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT2 035 |
| ✓ | BbhFormalFallaciesBk34decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT3 128 |
| ✓ | BbhFormalFallaciesBk35decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 822 |
| ✓ | BbhFormalFallaciesBk36decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT3 067 |
| ✓ | BbhFormalFallaciesBk37decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT2 647 |
| ✓ | BbhFormalFallaciesBk38decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 845 |
| ✓ | BbhFormalFallaciesBk39decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT2 974 |
| ✓ | BbhFormalFallaciesBk40decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT3 199 |
| ✓ | BbhFormalFallaciesBk41decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT1 156 |
| ✓ | BbhFormalFallaciesBk42decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT2 040 |
| ✓ | BbhFormalFallaciesBk43decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT2 316 |
| ✓ | BbhFormalFallaciesBk44decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 747 |
| ✓ | BbhFormalFallaciesBk45decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT1 044 |
| ✓ | BbhFormalFallaciesBk46decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT2 683 |
| ✓ | BbhFormalFallaciesBk47decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT2 358 |
| ✓ | BbhFormalFallaciesBk48decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT995 |
| ✓ | BbhFormalFallaciesBk49decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT2 982 |
| ✓ | BbhFormalFallaciesBk50decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT1 628 |
| ✓ | BbhFormalFallaciesBk51decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT1 917 |
| ✓ | BbhFormalFallaciesBk52decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT1 292 |
| ✓ | BbhFormalFallaciesBk53decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 828 |
| ✓ | BbhFormalFallaciesBk54decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT1 864 |
| ✓ | BbhFormalFallaciesBk55decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 274 |
| ✓ | BbhFormalFallaciesBk56decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT1 403 |
| ✓ | BbhFormalFallaciesBk57decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT2 029 |
| ✓ | BbhFormalFallaciesBk58decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT3 242 |
| ✓ | BbhFormalFallaciesBk59decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT2 554 |
| ✓ | BbhFormalFallaciesBk60decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT2 633 |
| ✓ | BbhFormalFallaciesBk61decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT2 389 |
| ✓ | BbhFormalFallaciesBk62decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT1 473 |
Reasoning - BBH formal fallacies (set 3)
32 passed |
100.0% |
IN34 560 OUT74 538 | |
| ✓ | BbhFormalFallaciesBk63decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT2 728 |
| ✓ | BbhFormalFallaciesBk64decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT1 718 |
| ✓ | BbhFormalFallaciesBk65decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT2 860 |
| ✓ | BbhFormalFallaciesBk66decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 174 |
| ✓ | BbhFormalFallaciesBk67decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 501 |
| ✓ | BbhFormalFallaciesBk68decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT3 581 |
| ✓ | BbhFormalFallaciesBk69decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT2 139 |
| ✓ | BbhFormalFallaciesBk70decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT1 726 |
| ✓ | BbhFormalFallaciesBk71decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 217 |
| ✓ | BbhFormalFallaciesBk72decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 794 |
| ✓ | BbhFormalFallaciesBk73decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 648 |
| ✓ | BbhFormalFallaciesBk74decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT3 181 |
| ✓ | BbhFormalFallaciesBk75decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT2 679 |
| ✓ | BbhFormalFallaciesBk76decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 756 |
| ✓ | BbhFormalFallaciesBk77decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT2 288 |
| ✓ | BbhFormalFallaciesBk78decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT3 200 |
| ✓ | BbhFormalFallaciesBk79decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT1 725 |
| ✓ | BbhFormalFallaciesBk80decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT5 309 |
| ✓ | BbhFormalFallaciesBk81decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT4 390 |
| ✓ | BbhFormalFallaciesBk82decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT1 729 |
| ✓ | BbhFormalFallaciesBk83decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT2 961 |
| ✓ | BbhFormalFallaciesBk84decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT781 |
| ✓ | BbhFormalFallaciesBk85decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 324 |
| ✓ | BbhFormalFallaciesBk86decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT2 493 |
| ✓ | BbhFormalFallaciesBk87decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT3 739 |
| ✓ | BbhFormalFallaciesBk88decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT2 148 |
| ✓ | BbhFormalFallaciesBk89decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT1 393 |
| ✓ | BbhFormalFallaciesBk90decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT2 806 |
| ✓ | BbhFormalFallaciesBk91decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT3 028 |
| ✓ | BbhFormalFallaciesBk92decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 750 |
| ✓ | BbhFormalFallaciesBk93decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 594 |
| ✓ | BbhFormalFallaciesBk94decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 178 |
Reasoning - BBH formal fallacies (set 4)
30 passed |
100.0% |
IN32 128 OUT61 553 | |
| ✓ | BbhFormalFallaciesBk100decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT2 745 |
| ✓ | BbhFormalFallaciesBk101decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT2 828 |
| ✓ | BbhFormalFallaciesBk102decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT1 250 |
| ✓ | BbhFormalFallaciesBk103decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 872 |
| ✓ | BbhFormalFallaciesBk104decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT2 633 |
| ✓ | BbhFormalFallaciesBk105decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT2 141 |
| ✓ | BbhFormalFallaciesBk106decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT1 864 |
| ✓ | BbhFormalFallaciesBk107decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT2 487 |
| ✓ | BbhFormalFallaciesBk108decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT2 085 |
| ✓ | BbhFormalFallaciesBk109decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 199 |
| ✓ | BbhFormalFallaciesBk110decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 811 |
| ✓ | BbhFormalFallaciesBk111decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT2 578 |
| ✓ | BbhFormalFallaciesBk112decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT1 183 |
| ✓ | BbhFormalFallaciesBk113decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT1 638 |
| ✓ | BbhFormalFallaciesBk114decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT2 193 |
| ✓ | BbhFormalFallaciesBk115decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 287 |
| ✓ | BbhFormalFallaciesBk116decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT1 150 |
| ✓ | BbhFormalFallaciesBk117decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 318 |
| ✓ | BbhFormalFallaciesBk118decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 268 |
| ✓ | BbhFormalFallaciesBk119decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 195 |
| ✓ | BbhFormalFallaciesBk120decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 903 |
| ✓ | BbhFormalFallaciesBk121decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 744 |
| ✓ | BbhFormalFallaciesBk122decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT2 445 |
| ✓ | BbhFormalFallaciesBk123decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 914 |
| ✓ | BbhFormalFallaciesBk124decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT1 961 |
| ✓ | BbhFormalFallaciesBk95decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT3 806 |
| ✓ | BbhFormalFallaciesBk96decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT2 511 |
| ✓ | BbhFormalFallaciesBk97decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT3 213 |
| ✓ | BbhFormalFallaciesBk98decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 225 |
| ✓ | BbhFormalFallaciesBk99decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT4 106 |
Reasoning - BBH formal fallacies (set 5)
32 passed |
100.0% |
IN34 430 OUT68 041 | |
| ✓ | BbhFormalFallacies125BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 584 |
| ✓ | BbhFormalFallacies126BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT2 470 |
| ✓ | BbhFormalFallacies127BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT1 094 |
| ✓ | BbhFormalFallacies128BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT1 484 |
| ✓ | BbhFormalFallacies129BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 043 |
| ✓ | BbhFormalFallacies130BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT1 327 |
| ✓ | BbhFormalFallacies131BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 607 |
| ✓ | BbhFormalFallacies132BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT2 463 |
| ✓ | BbhFormalFallacies133BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT882 |
| ✓ | BbhFormalFallacies134BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 041 OUT1 380 |
| ✓ | BbhFormalFallacies135BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT2 147 |
| ✓ | BbhFormalFallacies136BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT2 771 |
| ✓ | BbhFormalFallacies137BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 471 |
| ✓ | BbhFormalFallacies138BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT3 674 |
| ✓ | BbhFormalFallacies139BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 229 |
| ✓ | BbhFormalFallacies140BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 980 |
| ✓ | BbhFormalFallacies141BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 645 |
| ✓ | BbhFormalFallacies142BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT2 334 |
| ✓ | BbhFormalFallacies143BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 762 |
| ✓ | BbhFormalFallacies144BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT3 111 |
| ✓ | BbhFormalFallacies145BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 535 |
| ✓ | BbhFormalFallacies146BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT993 |
| ✓ | BbhFormalFallacies147BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT1 996 |
| ✓ | BbhFormalFallacies148BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT3 020 |
| ✓ | BbhFormalFallacies149BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT2 297 |
| ✓ | BbhFormalFallacies150BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT3 623 |
| ✓ | BbhFormalFallacies151BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 764 |
| ✓ | BbhFormalFallacies152BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT1 668 |
| ✓ | BbhFormalFallacies153BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT2 426 |
| ✓ | BbhFormalFallacies154BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT1 930 |
| ✓ | BbhFormalFallacies155BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT3 042 |
| ✓ | BbhFormalFallacies156BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT3 289 |
Reasoning - BBH formal fallacies (set 6)
32 passed |
100.0% |
IN34 425 OUT73 012 | |
| ✓ | BbhFormalFallacies157BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 838 |
| ✓ | BbhFormalFallacies158BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT2 010 |
| ✓ | BbhFormalFallacies159BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT3 896 |
| ✓ | BbhFormalFallacies160BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT2 190 |
| ✓ | BbhFormalFallacies161BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 837 |
| ✓ | BbhFormalFallacies162BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 565 |
| ✓ | BbhFormalFallacies163BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT2 321 |
| ✓ | BbhFormalFallacies164BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT4 088 |
| ✓ | BbhFormalFallacies165BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT3 769 |
| ✓ | BbhFormalFallacies166BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 855 |
| ✓ | BbhFormalFallacies167BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 711 |
| ✓ | BbhFormalFallacies168BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 041 OUT2 321 |
| ✓ | BbhFormalFallacies169BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT4 033 |
| ✓ | BbhFormalFallacies170BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT1 161 |
| ✓ | BbhFormalFallacies171BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT3 841 |
| ✓ | BbhFormalFallacies172BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT3 190 |
| ✓ | BbhFormalFallacies173BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 512 |
| ✓ | BbhFormalFallacies174BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT1 416 |
| ✓ | BbhFormalFallacies175BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT1 279 |
| ✓ | BbhFormalFallacies176BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 906 |
| ✓ | BbhFormalFallacies177BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT2 817 |
| ✓ | BbhFormalFallacies178BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT2 738 |
| ✓ | BbhFormalFallacies179BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT2 483 |
| ✓ | BbhFormalFallacies180BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT1 561 |
| ✓ | BbhFormalFallacies181BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 497 |
| ✓ | BbhFormalFallacies182BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT2 502 |
| ✓ | BbhFormalFallacies183BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 294 |
| ✓ | BbhFormalFallacies184BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT936 |
| ✓ | BbhFormalFallacies185BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 036 OUT1 152 |
| ✓ | BbhFormalFallacies186BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 211 |
| ✓ | BbhFormalFallacies187BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 123 OUT3 110 |
| ✓ | BbhFormalFallacies188BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT2 972 |
Reasoning - BBH formal fallacies (set 7)
32 passed |
100.0% |
IN34 299 OUT70 573 | |
| ✓ | BbhFormalFallacies189BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 073 |
| ✓ | BbhFormalFallacies190BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT2 144 |
| ✓ | BbhFormalFallacies191BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT945 |
| ✓ | BbhFormalFallacies192BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 196 |
| ✓ | BbhFormalFallacies193BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 226 |
| ✓ | BbhFormalFallacies194BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 356 |
| ✓ | BbhFormalFallacies195BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT1 493 |
| ✓ | BbhFormalFallacies196BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT2 455 |
| ✓ | BbhFormalFallacies197BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 377 |
| ✓ | BbhFormalFallacies198BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT2 924 |
| ✓ | BbhFormalFallacies199BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT2 742 |
| ✓ | BbhFormalFallacies200BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT1 706 |
| ✓ | BbhFormalFallacies201BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT3 270 |
| ✓ | BbhFormalFallacies202BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT2 732 |
| ✓ | BbhFormalFallacies203BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 033 |
| ✓ | BbhFormalFallacies204BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT802 |
| ✓ | BbhFormalFallacies205BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT4 284 |
| ✓ | BbhFormalFallacies206BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT4 062 |
| ✓ | BbhFormalFallacies207BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 797 |
| ✓ | BbhFormalFallacies208BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT4 155 |
| ✓ | BbhFormalFallacies209BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT2 498 |
| ✓ | BbhFormalFallacies210BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 141 |
| ✓ | BbhFormalFallacies211BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT3 565 |
| ✓ | BbhFormalFallacies212BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 057 |
| ✓ | BbhFormalFallacies213BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT2 831 |
| ✓ | BbhFormalFallacies214BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT2 588 |
| ✓ | BbhFormalFallacies215BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT2 241 |
| ✓ | BbhFormalFallacies216BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT3 079 |
| ✓ | BbhFormalFallacies217BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT2 210 |
| ✓ | BbhFormalFallacies218BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 744 |
| ✓ | BbhFormalFallacies219BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 147 |
| ✓ | BbhFormalFallacies220BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT1 700 |
Reasoning - BBH formal fallacies (set 8)
29 passed |
100.0% |
IN31 182 OUT71 284 | |
| ✓ | BbhFormalFallacies221BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT2 019 |
| ✓ | BbhFormalFallacies222BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 621 |
| ✓ | BbhFormalFallacies223BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT4 422 |
| ✓ | BbhFormalFallacies224BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT1 587 |
| ✓ | BbhFormalFallacies225BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT1 555 |
| ✓ | BbhFormalFallacies226BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT3 899 |
| ✓ | BbhFormalFallacies227BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT2 886 |
| ✓ | BbhFormalFallacies228BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT2 810 |
| ✓ | BbhFormalFallacies229BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT3 276 |
| ✓ | BbhFormalFallacies230BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT4 340 |
| ✓ | BbhFormalFallacies231BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT1 898 |
| ✓ | BbhFormalFallacies232BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 511 |
| ✓ | BbhFormalFallacies233BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 628 |
| ✓ | BbhFormalFallacies234BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT3 125 |
| ✓ | BbhFormalFallacies235BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT1 346 |
| ✓ | BbhFormalFallacies236BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 254 |
| ✓ | BbhFormalFallacies237BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT1 907 |
| ✓ | BbhFormalFallacies238BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 160 |
| ✓ | BbhFormalFallacies239BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT4 100 |
| ✓ | BbhFormalFallacies240BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT3 407 |
| ✓ | BbhFormalFallacies241BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT1 479 |
| ✓ | BbhFormalFallacies242BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT4 299 |
| ✓ | BbhFormalFallacies243BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT2 580 |
| ✓ | BbhFormalFallacies244BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT1 118 |
| ✓ | BbhFormalFallacies245BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT1 416 |
| ✓ | BbhFormalFallacies246BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT4 793 |
| ✓ | BbhFormalFallacies247BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT1 071 |
| ✓ | BbhFormalFallacies248BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT1 525 |
| ✓ | BbhFormalFallacies249BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 252 |
Reasoning - BBH geometric shapes (SVG paths) (set 1)
26 passed7 failed |
78.8% |
IN36 604 OUT84 793 | |
| ✓ | BbhGeometricShapes10BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT2 221 |
| ✓ | BbhGeometricShapes11BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT1 603 |
| × | BbhGeometricShapes12BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes12BL: expected (K), got [(A)] |
Failed | IN1 101 OUT2 253 |
| × | BbhGeometricShapes13BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes13BL: expected (K), got [(A)] |
Failed | IN1 097 OUT1 779 |
| × | BbhGeometricShapes14BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes14BL: expected (K), got [(H)] |
Failed | IN1 094 OUT3 712 |
| ✓ | BbhGeometricShapes15BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 975 |
| ✓ | BbhGeometricShapes16BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT3 637 |
| ✓ | BbhGeometricShapes17BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 194 OUT3 004 |
| × | BbhGeometricShapes18BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes18BL: expected (K), got [(H)] |
Failed | IN1 110 OUT3 505 |
| ✓ | BbhGeometricShapes19BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 538 |
| ✓ | BbhGeometricShapes20BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 447 |
| ✓ | BbhGeometricShapes21BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 168 OUT6 568 |
| ✓ | BbhGeometricShapes22BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 324 |
| ✓ | BbhGeometricShapes23BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 164 OUT2 570 |
| ✓ | BbhGeometricShapes24BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 168 OUT2 130 |
| ✓ | BbhGeometricShapes25BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT3 225 |
| ✓ | BbhGeometricShapes26BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT975 |
| ✓ | BbhGeometricShapes27BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT4 091 |
| ✓ | BbhGeometricShapes28BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT4 282 |
| × | BbhGeometricShapes29BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes29BL: expected (K), got [(H)] |
Failed | IN1 082 OUT1 350 |
| ✓ | BbhGeometricShapes30BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 599 |
| ✓ | BbhGeometricShapes31BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT2 493 |
| ✓ | BbhGeometricShapes32BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT2 447 |
| × | BbhGeometricShapes33BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes33BL: expected (K), got [(A)] |
Failed | IN1 101 OUT2 245 |
| ✓ | BbhGeometricShapes34BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT2 161 |
| ✓ | BbhGeometricShapes35BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT3 507 |
| ✓ | BbhGeometricShapes36BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT3 356 |
| ✓ | BbhGeometricShapes37BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 016 |
| ✓ | BbhGeometricShapes38BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 394 |
| ✓ | BbhGeometricShapes39BLThis SVG path element <path d="M 2.27,59.23 L 52.45,67.18"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT621 |
| ✓ | BbhGeometricShapes40BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 581 |
| × | BbhGeometricShapes8BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes8BL: expected (K), got [(H)] |
Failed | IN1 082 OUT4 351 |
| ✓ | BbhGeometricShapes9BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 833 |
Reasoning - BBH geometric shapes (SVG paths) (set 2)
25 passed7 failed |
78.1% |
IN35 033 OUT72 615 | |
| ✓ | BbhGeometricShapes41Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT3 140 |
| ✓ | BbhGeometricShapes42Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT3 544 |
| ✓ | BbhGeometricShapes43BmThis SVG path element <path d="M 98.96,92.35 L 44.27,28.27"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT686 |
| ✓ | BbhGeometricShapes44Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 103 |
| × | BbhGeometricShapes45Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes45Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 097 OUT1 931 |
| × | BbhGeometricShapes46Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes46Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT2 934 |
| ✓ | BbhGeometricShapes47Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 471 |
| ✓ | BbhGeometricShapes48Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 959 |
| ✓ | BbhGeometricShapes49Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 254 |
| ✓ | BbhGeometricShapes50Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 165 OUT1 657 |
| ✓ | BbhGeometricShapes51Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT3 548 |
| ✓ | BbhGeometricShapes52Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 714 |
| ✓ | BbhGeometricShapes53Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 190 |
| ✓ | BbhGeometricShapes54Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT3 422 |
| ✓ | BbhGeometricShapes55BmThis SVG path element <path d="M 32.95,20.92 L 31.65,35.07"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT429 |
| ✓ | BbhGeometricShapes56Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 173 |
| ✓ | BbhGeometricShapes57Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 839 |
| ✓ | BbhGeometricShapes58Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 748 |
| × | BbhGeometricShapes59Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes59Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 095 OUT2 006 |
| ✓ | BbhGeometricShapes60Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT3 142 |
| ✓ | BbhGeometricShapes61Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 675 |
| ✓ | BbhGeometricShapes62Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT3 120 |
| ✓ | BbhGeometricShapes63Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 506 |
| ✓ | BbhGeometricShapes64Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 065 |
| × | BbhGeometricShapes65Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes65Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT2 069 |
| × | BbhGeometricShapes66Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes66Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 095 OUT4 344 |
| ✓ | BbhGeometricShapes67Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT2 047 |
| ✓ | BbhGeometricShapes68BmThis SVG path element <path d="M 11.79,16.93 L 56.17,80.16"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT844 |
| ✓ | BbhGeometricShapes69Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 217 |
| ✓ | BbhGeometricShapes70Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT2 199 |
| × | BbhGeometricShapes71Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes71Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT3 223 |
| × | BbhGeometricShapes72Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes72Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 084 OUT1 416 |
Reasoning - BBH geometric shapes (SVG paths) (set 3)
27 passed5 failed |
84.4% |
IN34 974 OUT64 105 | |
| ✓ | BbhGeometricShapes100Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 607 |
| ✓ | BbhGeometricShapes101Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 118 |
| ✓ | BbhGeometricShapes102Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT2 026 |
| ✓ | BbhGeometricShapes103Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT2 573 |
| × | BbhGeometricShapes104Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes104Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 096 OUT2 079 |
| ✓ | BbhGeometricShapes73BmThis SVG path element <path d="M 31.97,19.98 L 38.55,43.43"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT697 |
| ✓ | BbhGeometricShapes74BmThis SVG path element <path d="M 81.33,89.86 L 65.20,41.90"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT787 |
| ✓ | BbhGeometricShapes75BmThis SVG path element <path d="M 27.49,66.87 L 19.65,56.41"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT719 |
| ✓ | BbhGeometricShapes76Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT2 183 |
| ✓ | BbhGeometricShapes77Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT3 821 |
| ✓ | BbhGeometricShapes78Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT2 879 |
| ✓ | BbhGeometricShapes79BmThis SVG path element <path d="M 14.31,45.36 L 17.22,37.11"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT575 |
| ✓ | BbhGeometricShapes80Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT2 802 |
| ✓ | BbhGeometricShapes81Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 168 OUT2 310 |
| ✓ | BbhGeometricShapes82Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 763 |
| ✓ | BbhGeometricShapes83Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 784 |
| ✓ | BbhGeometricShapes84Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 867 |
| ✓ | BbhGeometricShapes85BmThis SVG path element <path d="M 0.58,63.64 L 34.42,42.20"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT590 |
| ✓ | BbhGeometricShapes86Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 315 |
| × | BbhGeometricShapes87Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes87Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 084 OUT2 774 |
| ✓ | BbhGeometricShapes88Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT2 732 |
| ✓ | BbhGeometricShapes89Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 539 |
| ✓ | BbhGeometricShapes90Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT2 313 |
| ✓ | BbhGeometricShapes91Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 172 |
| ✓ | BbhGeometricShapes92Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 052 |
| × | BbhGeometricShapes93Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes93Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 084 OUT4 128 |
| ✓ | BbhGeometricShapes94Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 586 |
| × | BbhGeometricShapes95Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes95Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT2 345 |
| ✓ | BbhGeometricShapes96Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 514 |
| ✓ | BbhGeometricShapes97Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 571 |
| × | BbhGeometricShapes98Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes98Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 084 OUT1 248 |
| ✓ | BbhGeometricShapes99Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 636 |
Reasoning - BBH geometric shapes (SVG paths) (set 4)
24 passed8 failed |
75.0% |
IN35 266 OUT72 661 | |
| ✓ | BbhGeometricShapes105Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 164 OUT3 020 |
| ✓ | BbhGeometricShapes106Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT3 003 |
| ✓ | BbhGeometricShapes107BmThis SVG path element <path d="M 22.00,62.00 L 30.00,74.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT480 |
| ✓ | BbhGeometricShapes108Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 267 |
| × | BbhGeometricShapes109Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes109Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT2 326 |
| × | BbhGeometricShapes110Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes110Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 097 OUT1 637 |
| × | BbhGeometricShapes111Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes111Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 084 OUT1 811 |
| ✓ | BbhGeometricShapes112Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 324 |
| ✓ | BbhGeometricShapes113Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 141 OUT1 917 |
| ✓ | BbhGeometricShapes114Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT3 287 |
| ✓ | BbhGeometricShapes115Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT1 213 |
| ✓ | BbhGeometricShapes116Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 103 |
| ✓ | BbhGeometricShapes117Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 181 OUT2 364 |
| × | BbhGeometricShapes118Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes118Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 097 OUT3 205 |
| ✓ | BbhGeometricShapes119BmThis SVG path element <path d="M 17.11,37.49 L 43.48,13.53"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT451 |
| × | BbhGeometricShapes120Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes120Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 110 OUT3 100 |
| ✓ | BbhGeometricShapes121Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 788 |
| ✓ | BbhGeometricShapes122Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 234 |
| ✓ | BbhGeometricShapes123BmThis SVG path element <path d="M 51.00,18.00 L 5.00,62.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT497 |
| × | BbhGeometricShapes124Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes124Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT2 776 |
| ✓ | BbhGeometricShapes125Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT3 103 |
| ✓ | BbhGeometricShapes126Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT7 969 |
| ✓ | BbhGeometricShapes127Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 804 |
| × | BbhGeometricShapes128Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes128Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 097 OUT1 790 |
| ✓ | BbhGeometricShapes129Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 162 OUT2 045 |
| ✓ | BbhGeometricShapes130Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT2 964 |
| ✓ | BbhGeometricShapes131Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 207 OUT1 909 |
| ✓ | BbhGeometricShapes132BmThis SVG path element <path d="M 26.00,23.00 L 19.00,16.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT592 |
| ✓ | BbhGeometricShapes133Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT1 922 |
| × | BbhGeometricShapes134Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes134Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 084 OUT3 574 |
| ✓ | BbhGeometricShapes135Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 403 |
| ✓ | BbhGeometricShapes136BmThis SVG path element <path d="M 46.92,78.61 L 46.94,45.83"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT783 |
Reasoning - BBH geometric shapes (SVG paths) (set 5)
26 passed5 failed |
83.9% |
IN34 158 OUT73 077 | |
| ✓ | BbhGeometricShapes137Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT1 642 |
| ✓ | BbhGeometricShapes138Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT6 611 |
| ✓ | BbhGeometricShapes139BmThis SVG path element <path d="M 58.33,52.18 L 34.28,31.10"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT555 |
| ✓ | BbhGeometricShapes140Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 986 |
| ✓ | BbhGeometricShapes141BmThis SVG path element <path d="M 38.35,49.41 L 31.18,9.15"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT714 |
| × | BbhGeometricShapes142Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes142Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 097 OUT1 391 |
| ✓ | BbhGeometricShapes143Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 116 |
| ✓ | BbhGeometricShapes144Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 202 |
| × | BbhGeometricShapes145Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes145Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT1 849 |
| ✓ | BbhGeometricShapes146Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT2 390 |
| ✓ | BbhGeometricShapes147Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT3 562 |
| ✓ | BbhGeometricShapes148Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 956 |
| ✓ | BbhGeometricShapes149Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 958 |
| ✓ | BbhGeometricShapes150BmThis SVG path element <path d="M 90.41,82.38 L 45.53,97.84"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT842 |
| ✓ | BbhGeometricShapes151Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT2 279 |
| ✓ | BbhGeometricShapes152Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT4 867 |
| ✓ | BbhGeometricShapes153Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 618 |
| ✓ | BbhGeometricShapes154Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT965 |
| ✓ | BbhGeometricShapes155Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT2 770 |
| ✓ | BbhGeometricShapes156Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 995 |
| ✓ | BbhGeometricShapes157Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 526 |
| ✓ | BbhGeometricShapes158Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT2 546 |
| ✓ | BbhGeometricShapes159Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT1 866 |
| ✓ | BbhGeometricShapes160Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT2 713 |
| × | BbhGeometricShapes161Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes161Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT2 100 |
| ✓ | BbhGeometricShapes162Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 181 OUT3 340 |
| × | BbhGeometricShapes163Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes163Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 123 OUT3 573 |
| × | BbhGeometricShapes164Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes164Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 097 OUT1 971 |
| ✓ | BbhGeometricShapes165Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 523 |
| ✓ | BbhGeometricShapes166Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 443 |
| ✓ | BbhGeometricShapes167Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 168 OUT2 208 |
Reasoning - BBH geometric shapes (SVG paths) (set 6)
28 passed3 failed |
90.3% |
IN34 272 OUT76 078 | |
| ✓ | BbhGeometricShapes168Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 329 |
| ✓ | BbhGeometricShapes169Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 662 |
| ✓ | BbhGeometricShapes170Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT3 566 |
| ✓ | BbhGeometricShapes171Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 636 |
| ✓ | BbhGeometricShapes172Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT2 048 |
| ✓ | BbhGeometricShapes173Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT2 291 |
| ✓ | BbhGeometricShapes174Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 706 |
| ✓ | BbhGeometricShapes175Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT4 192 |
| ✓ | BbhGeometricShapes176Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT3 350 |
| ✓ | BbhGeometricShapes177Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT3 514 |
| ✓ | BbhGeometricShapes178Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT2 468 |
| ✓ | BbhGeometricShapes179Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 381 |
| ✓ | BbhGeometricShapes180Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 683 |
| × | BbhGeometricShapes181Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes181Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT2 087 |
| × | BbhGeometricShapes182Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes182Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 084 OUT3 757 |
| ✓ | BbhGeometricShapes183Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT3 219 |
| ✓ | BbhGeometricShapes184BmThis SVG path element <path d="M 25.00,38.00 L 89.00,58.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT718 |
| ✓ | BbhGeometricShapes185Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 985 |
| ✓ | BbhGeometricShapes186Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 914 |
| ✓ | BbhGeometricShapes187BmThis SVG path element <path d="M 51.78,63.32 L 1.41,97.49"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT802 |
| ✓ | BbhGeometricShapes188Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT3 889 |
| ✓ | BbhGeometricShapes189Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 909 |
| ✓ | BbhGeometricShapes190Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 168 OUT2 224 |
| ✓ | BbhGeometricShapes191Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 168 OUT1 957 |
| ✓ | BbhGeometricShapes192Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT2 469 |
| ✓ | BbhGeometricShapes193BmThis SVG path element <path d="M 41.63,86.23 L 86.15,83.17"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT622 |
| × | BbhGeometricShapes194Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes194Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 093 OUT3 367 |
| ✓ | BbhGeometricShapes195Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT2 055 |
| ✓ | BbhGeometricShapes196Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT4 278 |
| ✓ | BbhGeometricShapes197Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 694 |
| ✓ | BbhGeometricShapes198Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT3 306 |
Reasoning - BBH geometric shapes (SVG paths) (set 7)
23 passed9 failed |
71.9% |
IN35 108 OUT76 071 | |
| × | BbhGeometricShapes199BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes199BN: expected (K), got [(H)] |
Failed | IN1 084 OUT5 786 |
| × | BbhGeometricShapes200BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes200BN: expected (K), got [(A)] |
Failed | IN1 099 OUT2 589 |
| ✓ | BbhGeometricShapes201BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 849 |
| ✓ | BbhGeometricShapes202BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT1 110 |
| × | BbhGeometricShapes203BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes203BN: expected (K), got [(A)] |
Failed | IN1 097 OUT1 779 |
| ✓ | BbhGeometricShapes204BNThis SVG path element <path d="M 68.00,36.00 L 61.00,80.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT779 |
| ✓ | BbhGeometricShapes205BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 139 OUT2 472 |
| ✓ | BbhGeometricShapes206BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT1 921 |
| ✓ | BbhGeometricShapes207BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT2 091 |
| ✓ | BbhGeometricShapes208BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT4 262 |
| ✓ | BbhGeometricShapes209BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT5 150 |
| ✓ | BbhGeometricShapes210BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT3 723 |
| × | BbhGeometricShapes211BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes211BN: expected (K), got [(H)] |
Failed | IN1 123 OUT3 839 |
| ✓ | BbhGeometricShapes212BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT2 040 |
| ✓ | BbhGeometricShapes213BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 218 |
| ✓ | BbhGeometricShapes214BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 133 |
| ✓ | BbhGeometricShapes215BNThis SVG path element <path d="M 37.71,30.46 L 40.10,45.62"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT891 |
| ✓ | BbhGeometricShapes216BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT1 589 |
| ✓ | BbhGeometricShapes217BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 890 |
| ✓ | BbhGeometricShapes218BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT1 547 |
| ✓ | BbhGeometricShapes219BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 269 |
| × | BbhGeometricShapes220BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes220BN: expected (K), got [(A)] |
Failed | IN1 097 OUT2 038 |
| × | BbhGeometricShapes221BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes221BN: expected (K), got [(A)] |
Failed | IN1 101 OUT1 792 |
| ✓ | BbhGeometricShapes222BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 952 |
| × | BbhGeometricShapes223BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes223BN: expected (K), got [(H)] |
Failed | IN1 097 OUT2 973 |
| × | BbhGeometricShapes224BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes224BN: expected (K), got [(H)] |
Failed | IN1 084 OUT1 302 |
| ✓ | BbhGeometricShapes225BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT2 294 |
| × | BbhGeometricShapes226BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes226BN: expected (K), got [(H)] |
Failed | IN1 084 OUT3 872 |
| ✓ | BbhGeometricShapes227BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT1 444 |
| ✓ | BbhGeometricShapes228BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT2 477 |
| ✓ | BbhGeometricShapes229BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 301 |
| ✓ | BbhGeometricShapes230BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 699 |
Reasoning - BBH geometric shapes (SVG paths) (set 8)
29 passed3 failed |
90.6% |
IN33 687 OUT54 349 | |
| ✓ | BbhGeometricShapes231BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 149 |
| ✓ | BbhGeometricShapes232BNThis SVG path element <path d="M 93.43,69.32 L 68.06,88.58"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT754 |
| ✓ | BbhGeometricShapes233BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT2 724 |
| ✓ | BbhGeometricShapes234BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT2 509 |
| ✓ | BbhGeometricShapes235BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 041 |
| ✓ | BbhGeometricShapes236BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT3 677 |
| ✓ | BbhGeometricShapes237BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 492 |
| × | BbhGeometricShapes238BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes238BN: expected (K), got [(H)] |
Failed | IN1 084 OUT1 744 |
| ✓ | BbhGeometricShapes239BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT2 269 |
| ✓ | BbhGeometricShapes240BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 238 |
| ✓ | BbhGeometricShapes241BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT2 670 |
| × | BbhGeometricShapes242BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes242BN: expected (K), got [(A)] |
Failed | IN1 101 OUT1 838 |
| ✓ | BbhGeometricShapes243BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 777 |
| ✓ | BbhGeometricShapes244BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 405 |
| ✓ | BbhGeometricShapes245BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 514 |
| ✓ | BbhGeometricShapes246BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT3 014 |
| × | BbhGeometricShapes247BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes247BN: expected (K), got [(H)] |
Failed | IN1 093 OUT3 117 |
| ✓ | BbhGeometricShapes248BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 771 |
| ✓ | BbhGeometricShapes249BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT3 601 |
| ✓ | BbhHyperbaton10BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT827 |
| ✓ | BbhHyperbaton11BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT836 |
| ✓ | BbhHyperbaton12BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 688 |
| ✓ | BbhHyperbaton13BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT1 221 |
| ✓ | BbhHyperbaton14BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT636 |
| ✓ | BbhHyperbaton15BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT914 |
| ✓ | BbhHyperbaton16BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT600 |
| ✓ | BbhHyperbaton17BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT644 |
| ✓ | BbhHyperbaton18BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT719 |
| ✓ | BbhHyperbaton19BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT835 |
| ✓ | BbhHyperbaton20BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 067 |
| ✓ | BbhHyperbaton8BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 449 |
| ✓ | BbhHyperbaton9BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT609 |
Reasoning - BBH hyperbaton - adjective ordering (set 1)
32 passed |
100.0% |
IN31 442 OUT34 299 | |
| ✓ | BbhHyperbaton21BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT738 |
| ✓ | BbhHyperbaton22BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 676 |
| ✓ | BbhHyperbaton23BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT959 |
| ✓ | BbhHyperbaton24BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT539 |
| ✓ | BbhHyperbaton25BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT766 |
| ✓ | BbhHyperbaton26BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT884 |
| ✓ | BbhHyperbaton27BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 141 |
| ✓ | BbhHyperbaton28BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 137 |
| ✓ | BbhHyperbaton29BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 048 |
| ✓ | BbhHyperbaton30BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT916 |
| ✓ | BbhHyperbaton31BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 189 |
| ✓ | BbhHyperbaton32BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT659 |
| ✓ | BbhHyperbaton33BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 545 |
| ✓ | BbhHyperbaton34BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT856 |
| ✓ | BbhHyperbaton35BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 965 |
| ✓ | BbhHyperbaton36BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 092 |
| ✓ | BbhHyperbaton37BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT702 |
| ✓ | BbhHyperbaton38BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 346 |
| ✓ | BbhHyperbaton39BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT724 |
| ✓ | BbhHyperbaton40BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT936 |
| ✓ | BbhHyperbaton41BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT980 |
| ✓ | BbhHyperbaton42BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT517 |
| ✓ | BbhHyperbaton43BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT993 |
| ✓ | BbhHyperbaton44BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 187 |
| ✓ | BbhHyperbaton45BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 104 |
| ✓ | BbhHyperbaton46BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 447 |
| ✓ | BbhHyperbaton47BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT2 253 |
| ✓ | BbhHyperbaton48BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT818 |
| ✓ | BbhHyperbaton49BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 473 |
| ✓ | BbhHyperbaton50BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 125 |
| ✓ | BbhHyperbaton51BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT500 |
| ✓ | BbhHyperbaton52BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 084 |
Reasoning - BBH hyperbaton - adjective ordering (set 2)
32 passed |
100.0% |
IN31 460 OUT30 595 | |
| ✓ | BbhHyperbaton53BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT895 |
| ✓ | BbhHyperbaton54BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 252 |
| ✓ | BbhHyperbaton55BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 015 |
| ✓ | BbhHyperbaton56BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT898 |
| ✓ | BbhHyperbaton57BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 914 |
| ✓ | BbhHyperbaton58BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 703 |
| ✓ | BbhHyperbaton59BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT747 |
| ✓ | BbhHyperbaton60BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 072 |
| ✓ | BbhHyperbaton61BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT903 |
| ✓ | BbhHyperbaton62BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT638 |
| ✓ | BbhHyperbaton63BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT632 |
| ✓ | BbhHyperbaton64BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT806 |
| ✓ | BbhHyperbaton65BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT900 |
| ✓ | BbhHyperbaton66BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT567 |
| ✓ | BbhHyperbaton67BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 182 |
| ✓ | BbhHyperbaton68BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT637 |
| ✓ | BbhHyperbaton69BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 004 |
| ✓ | BbhHyperbaton70BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT782 |
| ✓ | BbhHyperbaton71BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 404 |
| ✓ | BbhHyperbaton72BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT1 163 |
| ✓ | BbhHyperbaton73BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT960 |
| ✓ | BbhHyperbaton74BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT485 |
| ✓ | BbhHyperbaton75BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT684 |
| ✓ | BbhHyperbaton76BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT537 |
| ✓ | BbhHyperbaton77BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT951 |
| ✓ | BbhHyperbaton78BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT557 |
| ✓ | BbhHyperbaton79BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 231 |
| ✓ | BbhHyperbaton80BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT787 |
| ✓ | BbhHyperbaton81BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 416 |
| ✓ | BbhHyperbaton82BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT838 |
| ✓ | BbhHyperbaton83BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 210 |
| ✓ | BbhHyperbaton84BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT825 |
Reasoning - BBH hyperbaton - adjective ordering (set 3)
30 passed |
100.0% |
IN29 494 OUT29 392 | |
| ✓ | BbhHyperbaton100BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT941 |
| ✓ | BbhHyperbaton101BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT715 |
| ✓ | BbhHyperbaton102BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 741 |
| ✓ | BbhHyperbaton103BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 254 |
| ✓ | BbhHyperbaton104BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT899 |
| ✓ | BbhHyperbaton105BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 066 |
| ✓ | BbhHyperbaton106BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT849 |
| ✓ | BbhHyperbaton107BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT909 |
| ✓ | BbhHyperbaton108BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT402 |
| ✓ | BbhHyperbaton109BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT810 |
| ✓ | BbhHyperbaton110BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT917 |
| ✓ | BbhHyperbaton111BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT349 |
| ✓ | BbhHyperbaton112BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT795 |
| ✓ | BbhHyperbaton113BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT834 |
| ✓ | BbhHyperbaton114BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT924 |
| ✓ | BbhHyperbaton85BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT653 |
| ✓ | BbhHyperbaton86BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 247 |
| ✓ | BbhHyperbaton87BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 380 |
| ✓ | BbhHyperbaton88BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 888 |
| ✓ | BbhHyperbaton89BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT660 |
| ✓ | BbhHyperbaton90BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT853 |
| ✓ | BbhHyperbaton91BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT763 |
| ✓ | BbhHyperbaton92BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 114 |
| ✓ | BbhHyperbaton93BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 375 |
| ✓ | BbhHyperbaton94BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT482 |
| ✓ | BbhHyperbaton95BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 748 |
| ✓ | BbhHyperbaton96BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT2 164 |
| ✓ | BbhHyperbaton97BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT531 |
| ✓ | BbhHyperbaton98BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT600 |
| ✓ | BbhHyperbaton99BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT529 |
Reasoning - BBH hyperbaton - adjective ordering (set 4)
32 passed |
100.0% |
IN31 426 OUT32 003 | |
| ✓ | BbhHyperbaton115BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 000 |
| ✓ | BbhHyperbaton116BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT724 |
| ✓ | BbhHyperbaton117BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT967 |
| ✓ | BbhHyperbaton118BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT565 |
| ✓ | BbhHyperbaton119BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT2 845 |
| ✓ | BbhHyperbaton120BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT364 |
| ✓ | BbhHyperbaton121BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT569 |
| ✓ | BbhHyperbaton122BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT916 |
| ✓ | BbhHyperbaton123BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT988 |
| ✓ | BbhHyperbaton124BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT935 |
| ✓ | BbhHyperbaton125BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT551 |
| ✓ | BbhHyperbaton126BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT2 059 |
| ✓ | BbhHyperbaton127BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 190 |
| ✓ | BbhHyperbaton128BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT729 |
| ✓ | BbhHyperbaton129BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT659 |
| ✓ | BbhHyperbaton130BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT780 |
| ✓ | BbhHyperbaton131BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT910 |
| ✓ | BbhHyperbaton132BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT2 129 |
| ✓ | BbhHyperbaton133BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT658 |
| ✓ | BbhHyperbaton134BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 633 |
| ✓ | BbhHyperbaton135BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT691 |
| ✓ | BbhHyperbaton136BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT859 |
| ✓ | BbhHyperbaton137BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT900 |
| ✓ | BbhHyperbaton138BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT603 |
| ✓ | BbhHyperbaton139BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT688 |
| ✓ | BbhHyperbaton140BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT676 |
| ✓ | BbhHyperbaton141BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT805 |
| ✓ | BbhHyperbaton142BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 137 |
| ✓ | BbhHyperbaton143BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT976 |
| ✓ | BbhHyperbaton144BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT931 |
| ✓ | BbhHyperbaton145BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 147 |
| ✓ | BbhHyperbaton146BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT1 419 |
Reasoning - BBH hyperbaton - adjective ordering (set 5)
32 passed |
100.0% |
IN31 448 OUT29 813 | |
| ✓ | BbhHyperbaton147BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 046 |
| ✓ | BbhHyperbaton148BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT766 |
| ✓ | BbhHyperbaton149BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 029 |
| ✓ | BbhHyperbaton150BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT805 |
| ✓ | BbhHyperbaton151BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 057 |
| ✓ | BbhHyperbaton152BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 133 |
| ✓ | BbhHyperbaton153BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 147 |
| ✓ | BbhHyperbaton154BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT928 |
| ✓ | BbhHyperbaton155BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 480 |
| ✓ | BbhHyperbaton156BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT994 |
| ✓ | BbhHyperbaton157BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT603 |
| ✓ | BbhHyperbaton158BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT942 |
| ✓ | BbhHyperbaton159BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT775 |
| ✓ | BbhHyperbaton160BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT759 |
| ✓ | BbhHyperbaton161BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 617 |
| ✓ | BbhHyperbaton162BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 215 |
| ✓ | BbhHyperbaton163BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT419 |
| ✓ | BbhHyperbaton164BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT937 |
| ✓ | BbhHyperbaton165BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT796 |
| ✓ | BbhHyperbaton166BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT778 |
| ✓ | BbhHyperbaton167BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT781 |
| ✓ | BbhHyperbaton168BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT573 |
| ✓ | BbhHyperbaton169BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT460 |
| ✓ | BbhHyperbaton170BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 369 |
| ✓ | BbhHyperbaton171BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 516 |
| ✓ | BbhHyperbaton172BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 144 |
| ✓ | BbhHyperbaton173BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT987 |
| ✓ | BbhHyperbaton174BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT696 |
| ✓ | BbhHyperbaton175BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT704 |
| ✓ | BbhHyperbaton176BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT547 |
| ✓ | BbhHyperbaton177BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT734 |
| ✓ | BbhHyperbaton178BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 076 |
Reasoning - BBH hyperbaton - adjective ordering (set 6)
32 passed |
100.0% |
IN31 436 OUT30 953 | |
| ✓ | BbhHyperbaton179BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT1 280 |
| ✓ | BbhHyperbaton180BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 072 |
| ✓ | BbhHyperbaton181BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 156 |
| ✓ | BbhHyperbaton182BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT701 |
| ✓ | BbhHyperbaton183BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 513 |
| ✓ | BbhHyperbaton184BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT931 |
| ✓ | BbhHyperbaton185BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT957 |
| ✓ | BbhHyperbaton186BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 280 |
| ✓ | BbhHyperbaton187BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 028 |
| ✓ | BbhHyperbaton188BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT562 |
| ✓ | BbhHyperbaton189BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT878 |
| ✓ | BbhHyperbaton190BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT565 |
| ✓ | BbhHyperbaton191BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT630 |
| ✓ | BbhHyperbaton192BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT2 196 |
| ✓ | BbhHyperbaton193BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 050 |
| ✓ | BbhHyperbaton194BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT436 |
| ✓ | BbhHyperbaton195BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT857 |
| ✓ | BbhHyperbaton196BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 033 |
| ✓ | BbhHyperbaton197BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT806 |
| ✓ | BbhHyperbaton198BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT734 |
| ✓ | BbhHyperbaton199BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT963 |
| ✓ | BbhHyperbaton200BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT379 |
| ✓ | BbhHyperbaton201BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT589 |
| ✓ | BbhHyperbaton202BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 601 |
| ✓ | BbhHyperbaton203BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT733 |
| ✓ | BbhHyperbaton204BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 258 |
| ✓ | BbhHyperbaton205BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 096 |
| ✓ | BbhHyperbaton206BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 198 |
| ✓ | BbhHyperbaton207BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 278 |
| ✓ | BbhHyperbaton208BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT620 |
| ✓ | BbhHyperbaton209BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT551 |
| ✓ | BbhHyperbaton210BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 022 |
Reasoning - BBH hyperbaton - adjective ordering (set 7)
32 passed |
100.0% |
IN31 422 OUT30 801 | |
| ✓ | BbhHyperbaton211BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT862 |
| ✓ | BbhHyperbaton212BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT960 |
| ✓ | BbhHyperbaton213BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 626 |
| ✓ | BbhHyperbaton214BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 857 |
| ✓ | BbhHyperbaton215BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 196 |
| ✓ | BbhHyperbaton216BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT967 |
| ✓ | BbhHyperbaton217BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT592 |
| ✓ | BbhHyperbaton218BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 076 |
| ✓ | BbhHyperbaton219BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 476 |
| ✓ | BbhHyperbaton220BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT546 |
| ✓ | BbhHyperbaton221BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT648 |
| ✓ | BbhHyperbaton222BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT976 |
| ✓ | BbhHyperbaton223BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT517 |
| ✓ | BbhHyperbaton224BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 298 |
| ✓ | BbhHyperbaton225BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT800 |
| ✓ | BbhHyperbaton226BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 121 |
| ✓ | BbhHyperbaton227BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT863 |
| ✓ | BbhHyperbaton228BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 089 |
| ✓ | BbhHyperbaton229BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT857 |
| ✓ | BbhHyperbaton230BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 479 |
| ✓ | BbhHyperbaton231BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT697 |
| ✓ | BbhHyperbaton232BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT843 |
| ✓ | BbhHyperbaton233BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT754 |
| ✓ | BbhHyperbaton234BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT835 |
| ✓ | BbhHyperbaton235BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT549 |
| ✓ | BbhHyperbaton236BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT879 |
| ✓ | BbhHyperbaton237BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT621 |
| ✓ | BbhHyperbaton238BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT550 |
| ✓ | BbhHyperbaton239BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 121 |
| ✓ | BbhHyperbaton240BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT942 |
| ✓ | BbhHyperbaton241BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 211 |
| ✓ | BbhHyperbaton242BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT993 |
Reasoning - BBH logic: boolean expressions, causal judgement, dates
22 passed2 failed |
91.7% |
IN25 847 OUT32 787 | |
| ✓ | BbhBooleanExpressionsBa0not ( True ) and ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT937 |
| ✓ | BbhBooleanExpressionsBa1True and not not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT802 |
| ✓ | BbhBooleanExpressionsBa2not True or False or ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT633 |
| ✓ | BbhBooleanExpressionsBa3False or not ( True ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT654 |
| ✓ | BbhBooleanExpressionsBa4True or not False and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT2 109 |
| ✓ | BbhBooleanExpressionsBa5False or not not not False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 013 |
| ✓ | BbhBooleanExpressionsBa6not True and ( False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT715 |
| ✓ | BbhBooleanExpressionsBa7True and not False or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT834 |
| ✓ | BbhCausalJudgementBa0judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 369 |
| ✓ | BbhCausalJudgementBa1judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 348 OUT1 904 |
| ✓ | BbhCausalJudgementBa2judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 350 OUT1 151 |
| ✓ | BbhCausalJudgementBa3judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 343 OUT3 555 |
| ✓ | BbhCausalJudgementBa4judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT3 015 |
| ✓ | BbhCausalJudgementBa5judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT2 594 |
| ✓ | BbhCausalJudgementBa6judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 527 |
| ✓ | BbhCausalJudgementBa7judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT1 737 |
| ✓ | BbhDateUnderstandingBa0date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT637 |
| ✓ | BbhDateUnderstandingBa1date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT860 |
| × | BbhDateUnderstandingBa2date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(B)")
Context: BbhDateUnderstandingBa2: expected (B), got [(C)] |
Failed | IN1 077 OUT936 |
| ✓ | BbhDateUnderstandingBa3date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT1 006 |
| ✓ | BbhDateUnderstandingBa4date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT882 |
| ✓ | BbhDateUnderstandingBa5date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 692 |
| × | BbhDateUnderstandingBa6date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhDateUnderstandingBa6: expected (A), got [(D)] |
Failed | IN1 085 OUT1 633 |
| ✓ | BbhDateUnderstandingBa7Tomorrow is 11/12/2019. What is the date yesterday in MM/DD/YYYY?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT592 |
Reasoning - BBH logic: disambiguation, dyck languages, formal fallacies
20 passed4 failed |
83.3% |
IN24 577 OUT57 714 | |
| ✓ | BbhDisambiguationQaBa0resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT645 |
| ✓ | BbhDisambiguationQaBa1resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT859 |
| × | BbhDisambiguationQaBa2resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhDisambiguationQaBa2: expected (C), got [(B)] |
Failed | IN1 024 OUT1 867 |
| ✓ | BbhDisambiguationQaBa3resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT1 609 |
| × | BbhDisambiguationQaBa4resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhDisambiguationQaBa4: expected (C), got [(A)] |
Failed | IN1 021 OUT1 956 |
| × | BbhDisambiguationQaBa5resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhDisambiguationQaBa5: expected (C), got [(B)] |
Failed | IN1 021 OUT1 375 |
| × | BbhDisambiguationQaBa6resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhDisambiguationQaBa6: expected (C), got [(A)] |
Failed | IN1 025 OUT1 358 |
| ✓ | BbhDisambiguationQaBa7resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 062 |
| ✓ | BbhDyckLanguagesBa0correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT827 |
| ✓ | BbhDyckLanguagesBa1correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT1 288 |
| ✓ | BbhDyckLanguagesBa2correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 036 OUT10 445 |
| ✓ | BbhDyckLanguagesBa3correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT2 458 |
| ✓ | BbhDyckLanguagesBa4correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT7 155 |
| ✓ | BbhDyckLanguagesBa5correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT2 202 |
| ✓ | BbhDyckLanguagesBa6correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT3 300 |
| ✓ | BbhDyckLanguagesBa7correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT2 436 |
| ✓ | BbhFormalFallaciesBa0decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT1 507 |
| ✓ | BbhFormalFallaciesBa1decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT948 |
| ✓ | BbhFormalFallaciesBa2decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT3 714 |
| ✓ | BbhFormalFallaciesBa3decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT2 045 |
| ✓ | BbhFormalFallaciesBa4decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT2 145 |
| ✓ | BbhFormalFallaciesBa5decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT3 397 |
| ✓ | BbhFormalFallaciesBa6decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 433 |
| ✓ | BbhFormalFallaciesBa7decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT1 683 |
Reasoning - BBH logic: geometric shapes, hyperbaton, logical deduction
22 passed2 failed |
91.7% |
IN25 418 OUT39 745 | |
| ✓ | BbhGeometricShapesBa0identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT2 303 |
| ✓ | BbhGeometricShapesBa1identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT1 928 |
| × | BbhGeometricShapesBa2identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapesBa2: expected (K), got [(H)] |
Failed | IN1 083 OUT4 091 |
| ✓ | BbhGeometricShapesBa3This SVG path element <path d="M 12.64,60.32 L 16.36,27.35"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT603 |
| × | BbhGeometricShapesBa4identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapesBa4: expected (K), got [(H)] |
Failed | IN1 110 OUT3 323 |
| ✓ | BbhGeometricShapesBa5identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT2 846 |
| ✓ | BbhGeometricShapesBa6identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 200 |
| ✓ | BbhGeometricShapesBa7identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT2 606 |
| ✓ | BbhHyperbatonBa0Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT638 |
| ✓ | BbhHyperbatonBa1Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT959 |
| ✓ | BbhHyperbatonBa2Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 161 |
| ✓ | BbhHyperbatonBa3Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT1 202 |
| ✓ | BbhHyperbatonBa4Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT770 |
| ✓ | BbhHyperbatonBa5Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT894 |
| ✓ | BbhHyperbatonBa6Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 132 |
| ✓ | BbhHyperbatonBa7Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT576 |
| ✓ | BbhLogicalDeductionFiveBa0order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 479 |
| ✓ | BbhLogicalDeductionFiveBa1order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT2 498 |
| ✓ | BbhLogicalDeductionFiveBa2order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 851 |
| ✓ | BbhLogicalDeductionFiveBa3order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 952 |
| ✓ | BbhLogicalDeductionFiveBa4order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 495 |
| ✓ | BbhLogicalDeductionFiveBa5order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 354 |
| ✓ | BbhLogicalDeductionFiveBa6order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT1 004 |
| ✓ | BbhLogicalDeductionFiveBa7order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 880 |
Reasoning - BBH logic: logical deduction, movie recommendation
24 passed |
100.0% |
IN25 463 OUT38 132 | |
| ✓ | BbhLogicalDeductionSeven0order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 774 |
| ✓ | BbhLogicalDeductionSeven1order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT1 983 |
| ✓ | BbhLogicalDeductionSeven2order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT2 592 |
| ✓ | BbhLogicalDeductionSeven3order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 976 |
| ✓ | BbhLogicalDeductionSeven4order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT1 639 |
| ✓ | BbhLogicalDeductionSeven5order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 095 OUT2 225 |
| ✓ | BbhLogicalDeductionSeven6order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT2 100 |
| ✓ | BbhLogicalDeductionSeven7order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 140 OUT2 020 |
| ✓ | BbhLogicalDeductionThree0order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT1 002 |
| ✓ | BbhLogicalDeductionThree1order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT727 |
| ✓ | BbhLogicalDeductionThree2order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT615 |
| ✓ | BbhLogicalDeductionThree3order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT465 |
| ✓ | BbhLogicalDeductionThree4order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT983 |
| ✓ | BbhLogicalDeductionThree5order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT797 |
| ✓ | BbhLogicalDeductionThree6order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT431 |
| ✓ | BbhLogicalDeductionThree7order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT457 |
| ✓ | BbhMovieRec0Find a movie similar to Batman, The Mask, The Fugitive, Pretty Woman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 166 |
| ✓ | BbhMovieRec1pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 060 |
| ✓ | BbhMovieRec2pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 780 |
| ✓ | BbhMovieRec3pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 541 |
| ✓ | BbhMovieRec4Find a movie similar to Braveheart, The Mask, The Fugitive, Batman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT5 465 |
| ✓ | BbhMovieRec5pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 630 |
| ✓ | BbhMovieRec6pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 429 |
| ✓ | BbhMovieRec7pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT2 275 |
Reasoning - BBH logic: multistep arithmetic, navigation, object counting
23 passed1 failed |
95.8% |
IN23 760 OUT33 910 | |
| ✓ | BbhMultistepArith0((-1 + 2 + 9 * 5) - (-2 + -4 + -4 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT876 |
| ✓ | BbhMultistepArith1((-9 * -5 - 6 + -2) - (-8 - -6 * -3 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT3 204 |
| ✓ | BbhMultistepArith2((3 * -3 * 6 + -5) - (-2 + -7 - 7 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 382 |
| ✓ | BbhMultistepArith3((6 * -6 * 8 * 1) * (-1 * 7 * -6 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT849 |
| ✓ | BbhMultistepArith4((-6 - -4 + 9 + 0) + (1 + -4 - -9 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT3 125 |
| ✓ | BbhMultistepArith5((-6 - 4 * 2 - 6) + (1 + -2 * 1 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT902 |
| ✓ | BbhMultistepArith6((1 - 0 + 1 - 4) - (-3 * 1 - -6 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT3 291 |
| ✓ | BbhMultistepArith7((1 + 7 * -9 + -5) + (3 + -5 * 2 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT950 |
| ✓ | BbhNavigate0does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT4 046 |
| ✓ | BbhNavigate1does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 795 |
| ✓ | BbhNavigate2does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 219 |
| ✓ | BbhNavigate3does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT976 |
| × | BbhNavigate4does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("No")
Context: BbhNavigate4: expected No, got [b]
Values:
got = "b" |
Failed | IN1 011 OUT3 139 |
| ✓ | BbhNavigate5does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT840 |
| ✓ | BbhNavigate6does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT3 297 |
| ✓ | BbhNavigate7does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT571 |
| ✓ | BbhObjectCount0count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT660 |
| ✓ | BbhObjectCount1count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT577 |
| ✓ | BbhObjectCount2I have a fridge, a chair, and a microwave. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT219 |
| ✓ | BbhObjectCount3count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT646 |
| ✓ | BbhObjectCount4count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT232 |
| ✓ | BbhObjectCount5I have a fish, a bear, and a frog. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT154 |
| ✓ | BbhObjectCount6I have a car, and a toaster. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT268 |
| ✓ | BbhObjectCount7count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT692 |
Reasoning - BBH logic: object tracking, web of lies, word sorting
23 passed1 failed |
95.8% |
IN24 382 OUT25 620 | |
| ✓ | BbhTrackThree0track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT707 |
| ✓ | BbhTrackThree1track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 388 |
| ✓ | BbhTrackThree2track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT895 |
| ✓ | BbhTrackThree3track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT833 |
| ✓ | BbhTrackThree4track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT795 |
| ✓ | BbhTrackThree5track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT928 |
| ✓ | BbhTrackThree6track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT939 |
| ✓ | BbhTrackThree7track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT608 |
| ✓ | BbhWebLies0evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT938 |
| ✓ | BbhWebLies1evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 155 |
| ✓ | BbhWebLies2evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT834 |
| ✓ | BbhWebLies3evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT767 |
| ✓ | BbhWebLies4evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 317 |
| ✓ | BbhWebLies5evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT826 |
| ✓ | BbhWebLies6evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 638 |
| ✓ | BbhWebLies7evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 063 |
| ✓ | BbhWordSort0Sort the following words alphabetically: List: syndrome therefrom1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN965 OUT472 |
| ✓ | BbhWordSort1sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 676 |
| ✓ | BbhWordSort2sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT941 |
| × | BbhWordSort3sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("fortescue helmsman percept purloin sioux")
Context: BbhWordSort3: expected sorted list, got [fortes cue, helms man, perc ept, pur loin, siou x.] |
Failed | IN973 OUT2 103 |
| ✓ | BbhWordSort4sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT463 |
| ✓ | BbhWordSort5sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 476 |
| ✓ | BbhWordSort6sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT1 412 |
| ✓ | BbhWordSort7sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT1 446 |
Reasoning - BBH logic: penguins table, colored objects, ruin names
24 passed |
100.0% |
IN25 960 OUT27 263 | |
| ✓ | BbhColoredObjects0reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT641 |
| ✓ | BbhColoredObjects1reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT1 350 |
| ✓ | BbhColoredObjects2reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT951 |
| ✓ | BbhColoredObjects3reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT1 789 |
| ✓ | BbhColoredObjects4reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT492 |
| ✓ | BbhColoredObjects5reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT1 199 |
| ✓ | BbhColoredObjects6reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT931 |
| ✓ | BbhColoredObjects7reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT384 |
| ✓ | BbhPenguins0answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 215 OUT368 |
| ✓ | BbhPenguins1answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT336 |
| ✓ | BbhPenguins2answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT641 |
| ✓ | BbhPenguins3answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 199 OUT791 |
| ✓ | BbhPenguins4answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT739 |
| ✓ | BbhPenguins5answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 205 OUT864 |
| ✓ | BbhPenguins6answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 224 OUT731 |
| ✓ | BbhPenguins7answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 200 OUT419 |
| ✓ | BbhRuinNames0pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 451 |
| ✓ | BbhRuinNames1pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 287 |
| ✓ | BbhRuinNames2pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT664 |
| ✓ | BbhRuinNames3pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 948 |
| ✓ | BbhRuinNames4pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT4 890 |
| ✓ | BbhRuinNames5pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT1 456 |
| ✓ | BbhRuinNames6pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 277 |
| ✓ | BbhRuinNames7pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 664 |
Reasoning - BBH logic: temporal sequences, object tracking
24 passed |
100.0% |
IN27 179 OUT36 846 | |
| ✓ | BbhTemporal0find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT594 |
| ✓ | BbhTemporal1find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 300 |
| ✓ | BbhTemporal2find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT1 082 |
| ✓ | BbhTemporal3find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 121 OUT1 216 |
| ✓ | BbhTemporal4find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 570 |
| ✓ | BbhTemporal5find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT779 |
| ✓ | BbhTemporal6find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT1 163 |
| ✓ | BbhTemporal7find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT1 150 |
| ✓ | BbhTrackFive0track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT3 003 |
| ✓ | BbhTrackFive1track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 295 |
| ✓ | BbhTrackFive2track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 415 |
| ✓ | BbhTrackFive3track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 300 |
| ✓ | BbhTrackFive4track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 417 |
| ✓ | BbhTrackFive5track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 156 |
| ✓ | BbhTrackFive6track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 675 |
| ✓ | BbhTrackFive7track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 042 |
| ✓ | BbhTrackSeven0track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 792 |
| ✓ | BbhTrackSeven1track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 632 |
| ✓ | BbhTrackSeven2track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 535 |
| ✓ | BbhTrackSeven3track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 520 |
| ✓ | BbhTrackSeven4track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 170 OUT1 832 |
| ✓ | BbhTrackSeven5track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 995 |
| ✓ | BbhTrackSeven6track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 973 |
| ✓ | BbhTrackSeven7track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 410 |
Reasoning - BBH logic: translation errors, snarks, sports
20 passed4 failed |
83.3% |
IN25 213 OUT29 195 | |
| ✓ | BbhSalientTransErr0identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 186 OUT1 119 |
| × | BbhSalientTransErr1identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhSalientTransErr1: expected (A), got [(D)] |
Failed | IN1 186 OUT1 108 |
| ✓ | BbhSalientTransErr2identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 183 OUT2 110 |
| × | BbhSalientTransErr3identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(F)")
Context: BbhSalientTransErr3: expected (F), got [(D)] |
Failed | IN1 180 OUT883 |
| × | BbhSalientTransErr4identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhSalientTransErr4: expected (A), got [(F)] |
Failed | IN1 177 OUT3 175 |
| ✓ | BbhSalientTransErr5identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 189 OUT2 275 |
| ✓ | BbhSalientTransErr6identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 199 OUT880 |
| ✓ | BbhSalientTransErr7identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 164 OUT1 852 |
| ✓ | BbhSnarks0Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT455 |
| ✓ | BbhSnarks1Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 941 |
| ✓ | BbhSnarks2Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT453 |
| ✓ | BbhSnarks3Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT777 |
| ✓ | BbhSnarks4Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 690 |
| ✓ | BbhSnarks5Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 045 |
| ✓ | BbhSnarks6Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT841 |
| ✓ | BbhSnarks7Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT931 |
| ✓ | BbhSports0Is the following sentence plausible? "Elias Lindholm beat the buzzer."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN969 OUT2 093 |
| ✓ | BbhSports1judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN968 OUT675 |
| ✓ | BbhSports2judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN972 OUT756 |
| ✓ | BbhSports3judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN976 OUT502 |
| ✓ | BbhSports4judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN970 OUT671 |
| ✓ | BbhSports5judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN976 OUT1 149 |
| × | BbhSports6Is the following sentence plausible? "David Silva took a throw in."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("yes")
Context: BbhSports6: expected yes, got [True] |
Failed | IN967 OUT879 |
| ✓ | BbhSports7judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN970 OUT935 |
Reasoning - BBH logical deduction - order 3 objects (set 1)
32 passed |
100.0% |
IN33 587 OUT28 263 | |
| ✓ | BbhLogicalDeductionThree21BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT727 |
| ✓ | BbhLogicalDeductionThree22BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT835 |
| ✓ | BbhLogicalDeductionThree23BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT626 |
| ✓ | BbhLogicalDeductionThree24BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT644 |
| ✓ | BbhLogicalDeductionThree25BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT1 566 |
| ✓ | BbhLogicalDeductionThree26BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT411 |
| ✓ | BbhLogicalDeductionThree27BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT1 238 |
| ✓ | BbhLogicalDeductionThree28BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT986 |
| ✓ | BbhLogicalDeductionThree29BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT610 |
| ✓ | BbhLogicalDeductionThree30BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT581 |
| ✓ | BbhLogicalDeductionThree31BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT365 |
| ✓ | BbhLogicalDeductionThree32BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 905 |
| ✓ | BbhLogicalDeductionThree33BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 077 |
| ✓ | BbhLogicalDeductionThree34BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT2 015 |
| ✓ | BbhLogicalDeductionThree35BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT893 |
| ✓ | BbhLogicalDeductionThree36BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT831 |
| ✓ | BbhLogicalDeductionThree37BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT811 |
| ✓ | BbhLogicalDeductionThree38BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 080 |
| ✓ | BbhLogicalDeductionThree39BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 159 |
| ✓ | BbhLogicalDeductionThree40BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT524 |
| ✓ | BbhLogicalDeductionThree41BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT349 |
| ✓ | BbhLogicalDeductionThree42BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT558 |
| ✓ | BbhLogicalDeductionThree43BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT492 |
| ✓ | BbhLogicalDeductionThree44BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT842 |
| ✓ | BbhLogicalDeductionThree45BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT888 |
| ✓ | BbhLogicalDeductionThree46BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT1 239 |
| ✓ | BbhLogicalDeductionThree47BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT927 |
| ✓ | BbhLogicalDeductionThree48BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT655 |
| ✓ | BbhLogicalDeductionThree49BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 094 |
| ✓ | BbhLogicalDeductionThree50BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT581 |
| ✓ | BbhLogicalDeductionThree51BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT543 |
| ✓ | BbhLogicalDeductionThree52BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT1 211 |
Reasoning - BBH logical deduction - order 3 objects (set 2)
32 passed |
100.0% |
IN33 436 OUT21 522 | |
| ✓ | BbhLogicalDeductionThree53BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT1 423 |
| ✓ | BbhLogicalDeductionThree54BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT782 |
| ✓ | BbhLogicalDeductionThree55BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT259 |
| ✓ | BbhLogicalDeductionThree56BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT413 |
| ✓ | BbhLogicalDeductionThree57BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT637 |
| ✓ | BbhLogicalDeductionThree58BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT527 |
| ✓ | BbhLogicalDeductionThree59BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT492 |
| ✓ | BbhLogicalDeductionThree60BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT658 |
| ✓ | BbhLogicalDeductionThree61BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT478 |
| ✓ | BbhLogicalDeductionThree62BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT979 |
| ✓ | BbhLogicalDeductionThree63BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT519 |
| ✓ | BbhLogicalDeductionThree64BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT819 |
| ✓ | BbhLogicalDeductionThree65BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT930 |
| ✓ | BbhLogicalDeductionThree66BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT494 |
| ✓ | BbhLogicalDeductionThree67BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT342 |
| ✓ | BbhLogicalDeductionThree68BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT424 |
| ✓ | BbhLogicalDeductionThree69BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT456 |
| ✓ | BbhLogicalDeductionThree70BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT656 |
| ✓ | BbhLogicalDeductionThree71BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT1 266 |
| ✓ | BbhLogicalDeductionThree72BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT363 |
| ✓ | BbhLogicalDeductionThree73BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT755 |
| ✓ | BbhLogicalDeductionThree74BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT804 |
| ✓ | BbhLogicalDeductionThree75BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT927 |
| ✓ | BbhLogicalDeductionThree76BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT1 198 |
| ✓ | BbhLogicalDeductionThree77BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT436 |
| ✓ | BbhLogicalDeductionThree78BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT369 |
| ✓ | BbhLogicalDeductionThree79BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT979 |
| ✓ | BbhLogicalDeductionThree80BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT828 |
| ✓ | BbhLogicalDeductionThree81BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT1 003 |
| ✓ | BbhLogicalDeductionThree82BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT843 |
| ✓ | BbhLogicalDeductionThree83BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT241 |
| ✓ | BbhLogicalDeductionThree84BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT222 |
Reasoning - BBH logical deduction - order 3 objects (set 3)
32 passed |
100.0% |
IN33 510 OUT24 525 | |
| ✓ | BbhLogicalDeductionThree100BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT993 |
| ✓ | BbhLogicalDeductionThree101BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT896 |
| ✓ | BbhLogicalDeductionThree102BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT379 |
| ✓ | BbhLogicalDeductionThree103BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT671 |
| ✓ | BbhLogicalDeductionThree104BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT975 |
| ✓ | BbhLogicalDeductionThree105BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT485 |
| ✓ | BbhLogicalDeductionThree106BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT796 |
| ✓ | BbhLogicalDeductionThree107BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT671 |
| ✓ | BbhLogicalDeductionThree108BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT490 |
| ✓ | BbhLogicalDeductionThree109BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 175 |
| ✓ | BbhLogicalDeductionThree110BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT919 |
| ✓ | BbhLogicalDeductionThree111BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT665 |
| ✓ | BbhLogicalDeductionThree112BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT502 |
| ✓ | BbhLogicalDeductionThree113BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT1 092 |
| ✓ | BbhLogicalDeductionThree114BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT295 |
| ✓ | BbhLogicalDeductionThree115BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT489 |
| ✓ | BbhLogicalDeductionThree116BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT1 305 |
| ✓ | BbhLogicalDeductionThree85BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT364 |
| ✓ | BbhLogicalDeductionThree86BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT655 |
| ✓ | BbhLogicalDeductionThree87BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT402 |
| ✓ | BbhLogicalDeductionThree88BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT1 487 |
| ✓ | BbhLogicalDeductionThree89BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT1 264 |
| ✓ | BbhLogicalDeductionThree90BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT755 |
| ✓ | BbhLogicalDeductionThree91BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT314 |
| ✓ | BbhLogicalDeductionThree92BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT573 |
| ✓ | BbhLogicalDeductionThree93BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 911 |
| ✓ | BbhLogicalDeductionThree94BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT794 |
| ✓ | BbhLogicalDeductionThree95BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT280 |
| ✓ | BbhLogicalDeductionThree96BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 268 |
| ✓ | BbhLogicalDeductionThree97BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT364 |
| ✓ | BbhLogicalDeductionThree98BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT701 |
| ✓ | BbhLogicalDeductionThree99BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT595 |
Reasoning - BBH logical deduction - order 3 objects (set 4)
32 passed |
100.0% |
IN33 558 OUT23 714 | |
| ✓ | BbhLogicalDeductionThree117BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT590 |
| ✓ | BbhLogicalDeductionThree118BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT850 |
| ✓ | BbhLogicalDeductionThree119BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT990 |
| ✓ | BbhLogicalDeductionThree120BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT910 |
| ✓ | BbhLogicalDeductionThree121BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT752 |
| ✓ | BbhLogicalDeductionThree122BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT669 |
| ✓ | BbhLogicalDeductionThree123BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT1 048 |
| ✓ | BbhLogicalDeductionThree124BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT457 |
| ✓ | BbhLogicalDeductionThree125BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT852 |
| ✓ | BbhLogicalDeductionThree126BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 016 |
| ✓ | BbhLogicalDeductionThree127BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT1 145 |
| ✓ | BbhLogicalDeductionThree128BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT885 |
| ✓ | BbhLogicalDeductionThree129BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT424 |
| ✓ | BbhLogicalDeductionThree130BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT693 |
| ✓ | BbhLogicalDeductionThree131BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT598 |
| ✓ | BbhLogicalDeductionThree132BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT648 |
| ✓ | BbhLogicalDeductionThree133BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT959 |
| ✓ | BbhLogicalDeductionThree134BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT321 |
| ✓ | BbhLogicalDeductionThree135BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT433 |
| ✓ | BbhLogicalDeductionThree136BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT926 |
| ✓ | BbhLogicalDeductionThree137BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 159 |
| ✓ | BbhLogicalDeductionThree138BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT354 |
| ✓ | BbhLogicalDeductionThree139BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT498 |
| ✓ | BbhLogicalDeductionThree140BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT992 |
| ✓ | BbhLogicalDeductionThree141BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT776 |
| ✓ | BbhLogicalDeductionThree142BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT701 |
| ✓ | BbhLogicalDeductionThree143BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT672 |
| ✓ | BbhLogicalDeductionThree144BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 041 |
| ✓ | BbhLogicalDeductionThree145BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT552 |
| ✓ | BbhLogicalDeductionThree146BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT466 |
| ✓ | BbhLogicalDeductionThree147BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT417 |
| ✓ | BbhLogicalDeductionThree148BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT920 |
Reasoning - BBH logical deduction - order 3 objects (set 5)
30 passed |
100.0% |
IN31 400 OUT23 707 | |
| ✓ | BbhLogicalDeductionThree149BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT642 |
| ✓ | BbhLogicalDeductionThree150BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT444 |
| ✓ | BbhLogicalDeductionThree151BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT672 |
| ✓ | BbhLogicalDeductionThree152BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT749 |
| ✓ | BbhLogicalDeductionThree153BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT560 |
| ✓ | BbhLogicalDeductionThree154BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT986 |
| ✓ | BbhLogicalDeductionThree155BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 492 |
| ✓ | BbhLogicalDeductionThree156BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT516 |
| ✓ | BbhLogicalDeductionThree157BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT682 |
| ✓ | BbhLogicalDeductionThree158BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT441 |
| ✓ | BbhLogicalDeductionThree159BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT771 |
| ✓ | BbhLogicalDeductionThree160BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT911 |
| ✓ | BbhLogicalDeductionThree161BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT564 |
| ✓ | BbhLogicalDeductionThree162BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT947 |
| ✓ | BbhLogicalDeductionThree163BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT772 |
| ✓ | BbhLogicalDeductionThree164BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT788 |
| ✓ | BbhLogicalDeductionThree165BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT1 125 |
| ✓ | BbhLogicalDeductionThree166BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT598 |
| ✓ | BbhLogicalDeductionThree167BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT622 |
| ✓ | BbhLogicalDeductionThree168BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT565 |
| ✓ | BbhLogicalDeductionThree169BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT556 |
| ✓ | BbhLogicalDeductionThree170BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT445 |
| ✓ | BbhLogicalDeductionThree171BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 376 |
| ✓ | BbhLogicalDeductionThree172BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 673 |
| ✓ | BbhLogicalDeductionThree173BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT1 120 |
| ✓ | BbhLogicalDeductionThree174BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT514 |
| ✓ | BbhLogicalDeductionThree175BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT765 |
| ✓ | BbhLogicalDeductionThree176BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT407 |
| ✓ | BbhLogicalDeductionThree177BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 377 |
| ✓ | BbhLogicalDeductionThree178BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT627 |
Reasoning - BBH logical deduction - order 3 objects (set 6)
32 passed |
100.0% |
IN33 455 OUT22 959 | |
| ✓ | BbhLogicalDeductionThreeObjects179BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT1 624 |
| ✓ | BbhLogicalDeductionThreeObjects180BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT413 |
| ✓ | BbhLogicalDeductionThreeObjects181BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT717 |
| ✓ | BbhLogicalDeductionThreeObjects182BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT371 |
| ✓ | BbhLogicalDeductionThreeObjects183BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT762 |
| ✓ | BbhLogicalDeductionThreeObjects184BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT848 |
| ✓ | BbhLogicalDeductionThreeObjects185BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT440 |
| ✓ | BbhLogicalDeductionThreeObjects186BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT582 |
| ✓ | BbhLogicalDeductionThreeObjects187BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT790 |
| ✓ | BbhLogicalDeductionThreeObjects188BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT854 |
| ✓ | BbhLogicalDeductionThreeObjects189BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT358 |
| ✓ | BbhLogicalDeductionThreeObjects190BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT991 |
| ✓ | BbhLogicalDeductionThreeObjects191BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT628 |
| ✓ | BbhLogicalDeductionThreeObjects192BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT349 |
| ✓ | BbhLogicalDeductionThreeObjects193BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT345 |
| ✓ | BbhLogicalDeductionThreeObjects194BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT832 |
| ✓ | BbhLogicalDeductionThreeObjects195BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT855 |
| ✓ | BbhLogicalDeductionThreeObjects196BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 025 |
| ✓ | BbhLogicalDeductionThreeObjects197BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 022 |
| ✓ | BbhLogicalDeductionThreeObjects198BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT1 181 |
| ✓ | BbhLogicalDeductionThreeObjects199BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT571 |
| ✓ | BbhLogicalDeductionThreeObjects200BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT1 314 |
| ✓ | BbhLogicalDeductionThreeObjects201BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT279 |
| ✓ | BbhLogicalDeductionThreeObjects202BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT1 005 |
| ✓ | BbhLogicalDeductionThreeObjects203BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT347 |
| ✓ | BbhLogicalDeductionThreeObjects204BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT362 |
| ✓ | BbhLogicalDeductionThreeObjects205BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT510 |
| ✓ | BbhLogicalDeductionThreeObjects206BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT671 |
| ✓ | BbhLogicalDeductionThreeObjects207BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT328 |
| ✓ | BbhLogicalDeductionThreeObjects208BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT408 |
| ✓ | BbhLogicalDeductionThreeObjects209BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT1 035 |
| ✓ | BbhLogicalDeductionThreeObjects210BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT1 142 |
Reasoning - BBH logical deduction - order 3 objects (set 7)
32 passed |
100.0% |
IN33 488 OUT24 142 | |
| ✓ | BbhLogicalDeductionThreeObjects211BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT682 |
| ✓ | BbhLogicalDeductionThreeObjects212BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT462 |
| ✓ | BbhLogicalDeductionThreeObjects213BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT1 078 |
| ✓ | BbhLogicalDeductionThreeObjects214BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT545 |
| ✓ | BbhLogicalDeductionThreeObjects215BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT643 |
| ✓ | BbhLogicalDeductionThreeObjects216BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 170 |
| ✓ | BbhLogicalDeductionThreeObjects217BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT652 |
| ✓ | BbhLogicalDeductionThreeObjects218BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT725 |
| ✓ | BbhLogicalDeductionThreeObjects219BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT795 |
| ✓ | BbhLogicalDeductionThreeObjects220BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT374 |
| ✓ | BbhLogicalDeductionThreeObjects221BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT272 |
| ✓ | BbhLogicalDeductionThreeObjects222BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT787 |
| ✓ | BbhLogicalDeductionThreeObjects223BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT389 |
| ✓ | BbhLogicalDeductionThreeObjects224BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT685 |
| ✓ | BbhLogicalDeductionThreeObjects225BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT840 |
| ✓ | BbhLogicalDeductionThreeObjects226BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT697 |
| ✓ | BbhLogicalDeductionThreeObjects227BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT548 |
| ✓ | BbhLogicalDeductionThreeObjects228BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT607 |
| ✓ | BbhLogicalDeductionThreeObjects229BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT996 |
| ✓ | BbhLogicalDeductionThreeObjects230BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT467 |
| ✓ | BbhLogicalDeductionThreeObjects231BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 079 |
| ✓ | BbhLogicalDeductionThreeObjects232BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT1 332 |
| ✓ | BbhLogicalDeductionThreeObjects233BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT1 386 |
| ✓ | BbhLogicalDeductionThreeObjects234BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT752 |
| ✓ | BbhLogicalDeductionThreeObjects235BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT936 |
| ✓ | BbhLogicalDeductionThreeObjects236BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT528 |
| ✓ | BbhLogicalDeductionThreeObjects237BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT563 |
| ✓ | BbhLogicalDeductionThreeObjects238BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT1 192 |
| ✓ | BbhLogicalDeductionThreeObjects239BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT992 |
| ✓ | BbhLogicalDeductionThreeObjects240BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT648 |
| ✓ | BbhLogicalDeductionThreeObjects241BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT439 |
| ✓ | BbhLogicalDeductionThreeObjects242BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT881 |
Reasoning - BBH logical deduction - order 5 objects (set 1)
30 passed |
100.0% |
IN31 850 OUT37 472 | |
| ✓ | BbhHyperbaton243BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT855 |
| ✓ | BbhHyperbaton244BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT765 |
| ✓ | BbhHyperbaton245BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT607 |
| ✓ | BbhHyperbaton246BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT734 |
| ✓ | BbhHyperbaton247BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT968 |
| ✓ | BbhHyperbaton248BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT744 |
| ✓ | BbhHyperbaton249BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 480 |
| ✓ | BbhLogicalDeductionFiveObjects10BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 245 |
| ✓ | BbhLogicalDeductionFiveObjects11BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 228 |
| ✓ | BbhLogicalDeductionFiveObjects12BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 339 |
| ✓ | BbhLogicalDeductionFiveObjects13BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 865 |
| ✓ | BbhLogicalDeductionFiveObjects14BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT676 |
| ✓ | BbhLogicalDeductionFiveObjects15BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT951 |
| ✓ | BbhLogicalDeductionFiveObjects16BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT2 053 |
| ✓ | BbhLogicalDeductionFiveObjects17BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 372 |
| ✓ | BbhLogicalDeductionFiveObjects18BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT751 |
| ✓ | BbhLogicalDeductionFiveObjects19BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT2 249 |
| ✓ | BbhLogicalDeductionFiveObjects20BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 044 |
| ✓ | BbhLogicalDeductionFiveObjects21BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 444 |
| ✓ | BbhLogicalDeductionFiveObjects22BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 149 |
| ✓ | BbhLogicalDeductionFiveObjects23BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 475 |
| ✓ | BbhLogicalDeductionFiveObjects24BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT998 |
| ✓ | BbhLogicalDeductionFiveObjects25BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT2 002 |
| ✓ | BbhLogicalDeductionFiveObjects26BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 389 |
| ✓ | BbhLogicalDeductionFiveObjects27BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT1 906 |
| ✓ | BbhLogicalDeductionFiveObjects28BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 186 |
| ✓ | BbhLogicalDeductionFiveObjects29BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT851 |
| ✓ | BbhLogicalDeductionFiveObjects30BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT748 |
| ✓ | BbhLogicalDeductionFiveObjects8BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 759 |
| ✓ | BbhLogicalDeductionFiveObjects9BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 639 |
Reasoning - BBH logical deduction - order 5 objects (set 2)
32 passed |
100.0% |
IN34 885 OUT52 470 | |
| ✓ | BbhLogicalDeductionFive189BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 202 |
| ✓ | BbhLogicalDeductionFive190BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 639 |
| ✓ | BbhLogicalDeductionFive191BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 375 |
| ✓ | BbhLogicalDeductionFive192BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT1 684 |
| ✓ | BbhLogicalDeductionFive193BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 311 |
| ✓ | BbhLogicalDeductionFive194BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 166 |
| ✓ | BbhLogicalDeductionFive195BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 245 |
| ✓ | BbhLogicalDeductionFive196BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT2 134 |
| ✓ | BbhLogicalDeductionFive197BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 458 |
| ✓ | BbhLogicalDeductionFive198BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 649 |
| ✓ | BbhLogicalDeductionFive199BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT2 058 |
| ✓ | BbhLogicalDeductionFive200BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT1 395 |
| ✓ | BbhLogicalDeductionFive201BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 689 |
| ✓ | BbhLogicalDeductionFive202BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 104 |
| ✓ | BbhLogicalDeductionFive203BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 757 |
| ✓ | BbhLogicalDeductionFive204BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT2 147 |
| ✓ | BbhLogicalDeductionFive205BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT2 164 |
| ✓ | BbhLogicalDeductionFive206BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 455 |
| ✓ | BbhLogicalDeductionFive207BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 265 |
| ✓ | BbhLogicalDeductionFive208BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT2 073 |
| ✓ | BbhLogicalDeductionFive209BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 512 |
| ✓ | BbhLogicalDeductionFive210BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 312 |
| ✓ | BbhLogicalDeductionFive211BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 289 |
| ✓ | BbhLogicalDeductionFive212BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT651 |
| ✓ | BbhLogicalDeductionFive213BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 680 |
| ✓ | BbhLogicalDeductionFive214BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 357 |
| ✓ | BbhLogicalDeductionFive215BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT2 743 |
| ✓ | BbhLogicalDeductionFive216BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 123 |
| ✓ | BbhLogicalDeductionFive217BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 747 |
| ✓ | BbhLogicalDeductionFive218BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT420 |
| ✓ | BbhLogicalDeductionFive219BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT3 034 |
| ✓ | BbhLogicalDeductionFive220BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 632 |
Reasoning - BBH logical deduction - order 5 objects (set 3)
29 passed |
100.0% |
IN31 705 OUT41 561 | |
| ✓ | BbhLogicalDeductionFive221BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 798 |
| ✓ | BbhLogicalDeductionFive222BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 256 |
| ✓ | BbhLogicalDeductionFive223BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 922 |
| ✓ | BbhLogicalDeductionFive224BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 359 |
| ✓ | BbhLogicalDeductionFive225BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 553 |
| ✓ | BbhLogicalDeductionFive226BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 820 |
| ✓ | BbhLogicalDeductionFive227BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 655 |
| ✓ | BbhLogicalDeductionFive228BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 536 |
| ✓ | BbhLogicalDeductionFive229BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 041 |
| ✓ | BbhLogicalDeductionFive230BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT749 |
| ✓ | BbhLogicalDeductionFive231BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 521 |
| ✓ | BbhLogicalDeductionFive232BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT623 |
| ✓ | BbhLogicalDeductionFive233BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 545 |
| ✓ | BbhLogicalDeductionFive234BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT2 252 |
| ✓ | BbhLogicalDeductionFive235BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 765 |
| ✓ | BbhLogicalDeductionFive236BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 436 |
| ✓ | BbhLogicalDeductionFive237BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT926 |
| ✓ | BbhLogicalDeductionFive238BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 493 |
| ✓ | BbhLogicalDeductionFive239BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 127 |
| ✓ | BbhLogicalDeductionFive240BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT3 086 |
| ✓ | BbhLogicalDeductionFive241BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 435 |
| ✓ | BbhLogicalDeductionFive242BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 188 |
| ✓ | BbhLogicalDeductionFive243BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 652 |
| ✓ | BbhLogicalDeductionFive244BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT800 |
| ✓ | BbhLogicalDeductionFive245BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT939 |
| ✓ | BbhLogicalDeductionFive246BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT1 440 |
| ✓ | BbhLogicalDeductionFive247BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 018 |
| ✓ | BbhLogicalDeductionFive248BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 215 |
| ✓ | BbhLogicalDeductionFive249BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 411 |
Reasoning - BBH logical deduction - order 5 objects (set 4)
32 passed |
100.0% |
IN35 046 OUT51 703 | |
| ✓ | BbhLogicalDeductionFiveObjects31BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 407 |
| ✓ | BbhLogicalDeductionFiveObjects32BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 095 |
| ✓ | BbhLogicalDeductionFiveObjects33BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 139 |
| ✓ | BbhLogicalDeductionFiveObjects34BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT2 732 |
| ✓ | BbhLogicalDeductionFiveObjects35BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 439 |
| ✓ | BbhLogicalDeductionFiveObjects36BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT915 |
| ✓ | BbhLogicalDeductionFiveObjects37BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 686 |
| ✓ | BbhLogicalDeductionFiveObjects38BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT1 205 |
| ✓ | BbhLogicalDeductionFiveObjects39BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 239 |
| ✓ | BbhLogicalDeductionFiveObjects40BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT1 309 |
| ✓ | BbhLogicalDeductionFiveObjects41BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT2 571 |
| ✓ | BbhLogicalDeductionFiveObjects42BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT694 |
| ✓ | BbhLogicalDeductionFiveObjects43BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 485 |
| ✓ | BbhLogicalDeductionFiveObjects44BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 810 |
| ✓ | BbhLogicalDeductionFiveObjects45BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT2 400 |
| ✓ | BbhLogicalDeductionFiveObjects46BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 481 |
| ✓ | BbhLogicalDeductionFiveObjects47BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT2 682 |
| ✓ | BbhLogicalDeductionFiveObjects48BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT856 |
| ✓ | BbhLogicalDeductionFiveObjects49BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 308 |
| ✓ | BbhLogicalDeductionFiveObjects50BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT2 460 |
| ✓ | BbhLogicalDeductionFiveObjects51BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT2 928 |
| ✓ | BbhLogicalDeductionFiveObjects52BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT452 |
| ✓ | BbhLogicalDeductionFiveObjects53BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 862 |
| ✓ | BbhLogicalDeductionFiveObjects54BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 837 |
| ✓ | BbhLogicalDeductionFiveObjects55BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 447 |
| ✓ | BbhLogicalDeductionFiveObjects56BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 806 |
| ✓ | BbhLogicalDeductionFiveObjects57BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 283 |
| ✓ | BbhLogicalDeductionFiveObjects58BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 333 |
| ✓ | BbhLogicalDeductionFiveObjects59BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 848 |
| ✓ | BbhLogicalDeductionFiveObjects60BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT2 120 |
| ✓ | BbhLogicalDeductionFiveObjects61BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 068 |
| ✓ | BbhLogicalDeductionFiveObjects62BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 806 |
Reasoning - BBH logical deduction - order 5 objects (set 5)
32 passed |
100.0% |
IN35 054 OUT59 654 | |
| ✓ | BbhLogicalDeductionFiveObjects63BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 206 |
| ✓ | BbhLogicalDeductionFiveObjects64BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 704 |
| ✓ | BbhLogicalDeductionFiveObjects65BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT4 152 |
| ✓ | BbhLogicalDeductionFiveObjects66BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT7 863 |
| ✓ | BbhLogicalDeductionFiveObjects67BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 964 |
| ✓ | BbhLogicalDeductionFiveObjects68BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 193 |
| ✓ | BbhLogicalDeductionFiveObjects69BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 468 |
| ✓ | BbhLogicalDeductionFiveObjects70BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 477 |
| ✓ | BbhLogicalDeductionFiveObjects71BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT947 |
| ✓ | BbhLogicalDeductionFiveObjects72BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT2 203 |
| ✓ | BbhLogicalDeductionFiveObjects73BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 216 |
| ✓ | BbhLogicalDeductionFiveObjects74BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 101 |
| ✓ | BbhLogicalDeductionFiveObjects75BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 742 |
| ✓ | BbhLogicalDeductionFiveObjects76BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 699 |
| ✓ | BbhLogicalDeductionFiveObjects77BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 605 |
| ✓ | BbhLogicalDeductionFiveObjects78BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 379 |
| ✓ | BbhLogicalDeductionFiveObjects79BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT707 |
| ✓ | BbhLogicalDeductionFiveObjects80BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 628 |
| ✓ | BbhLogicalDeductionFiveObjects81BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 520 |
| ✓ | BbhLogicalDeductionFiveObjects82BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT2 427 |
| ✓ | BbhLogicalDeductionFiveObjects83BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 086 |
| ✓ | BbhLogicalDeductionFiveObjects84BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 414 |
| ✓ | BbhLogicalDeductionFiveObjects85BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT1 486 |
| ✓ | BbhLogicalDeductionFiveObjects86BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 235 |
| ✓ | BbhLogicalDeductionFiveObjects87BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 553 |
| ✓ | BbhLogicalDeductionFiveObjects88BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT1 208 |
| ✓ | BbhLogicalDeductionFiveObjects89BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 510 |
| ✓ | BbhLogicalDeductionFiveObjects90BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 683 |
| ✓ | BbhLogicalDeductionFiveObjects91BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 155 |
| ✓ | BbhLogicalDeductionFiveObjects92BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT2 690 |
| ✓ | BbhLogicalDeductionFiveObjects93BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT2 817 |
| ✓ | BbhLogicalDeductionFiveObjects94BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT1 616 |
Reasoning - BBH logical deduction - order 5 objects (set 6)
32 passed |
100.0% |
IN34 988 OUT50 603 | |
| ✓ | BbhLogicalDeductionFiveObjects100BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT2 209 |
| ✓ | BbhLogicalDeductionFiveObjects101BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT2 013 |
| ✓ | BbhLogicalDeductionFiveObjects102BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT2 172 |
| ✓ | BbhLogicalDeductionFiveObjects103BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 841 |
| ✓ | BbhLogicalDeductionFiveObjects104BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 414 |
| ✓ | BbhLogicalDeductionFiveObjects105BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 579 |
| ✓ | BbhLogicalDeductionFiveObjects106BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT2 113 |
| ✓ | BbhLogicalDeductionFiveObjects107BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 125 |
| ✓ | BbhLogicalDeductionFiveObjects108BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 923 |
| ✓ | BbhLogicalDeductionFiveObjects109BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 986 |
| ✓ | BbhLogicalDeductionFiveObjects110BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT2 156 |
| ✓ | BbhLogicalDeductionFiveObjects111BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 048 |
| ✓ | BbhLogicalDeductionFiveObjects112BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 928 |
| ✓ | BbhLogicalDeductionFiveObjects113BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 554 |
| ✓ | BbhLogicalDeductionFiveObjects114BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 943 |
| ✓ | BbhLogicalDeductionFiveObjects115BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 307 |
| ✓ | BbhLogicalDeductionFiveObjects116BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT1 596 |
| ✓ | BbhLogicalDeductionFiveObjects117BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT2 241 |
| ✓ | BbhLogicalDeductionFiveObjects118BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 361 |
| ✓ | BbhLogicalDeductionFiveObjects119BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 812 |
| ✓ | BbhLogicalDeductionFiveObjects120BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT2 525 |
| ✓ | BbhLogicalDeductionFiveObjects121BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT888 |
| ✓ | BbhLogicalDeductionFiveObjects122BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 101 |
| ✓ | BbhLogicalDeductionFiveObjects123BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 251 |
| ✓ | BbhLogicalDeductionFiveObjects124BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT858 |
| ✓ | BbhLogicalDeductionFiveObjects125BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 277 |
| ✓ | BbhLogicalDeductionFiveObjects126BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT2 000 |
| ✓ | BbhLogicalDeductionFiveObjects95BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 153 |
| ✓ | BbhLogicalDeductionFiveObjects96BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 052 |
| ✓ | BbhLogicalDeductionFiveObjects97BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 712 |
| ✓ | BbhLogicalDeductionFiveObjects98BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT542 |
| ✓ | BbhLogicalDeductionFiveObjects99BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT923 |
Reasoning - BBH logical deduction - order 5 objects (set 7)
32 passed |
100.0% |
IN35 021 OUT51 555 | |
| ✓ | BbhLogicalDeductionFiveObjects127BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 640 |
| ✓ | BbhLogicalDeductionFiveObjects128BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 612 |
| ✓ | BbhLogicalDeductionFiveObjects129BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT2 226 |
| ✓ | BbhLogicalDeductionFiveObjects130BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 945 |
| ✓ | BbhLogicalDeductionFiveObjects131BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT2 170 |
| ✓ | BbhLogicalDeductionFiveObjects132BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 853 |
| ✓ | BbhLogicalDeductionFiveObjects133BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 184 |
| ✓ | BbhLogicalDeductionFiveObjects134BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT1 095 |
| ✓ | BbhLogicalDeductionFiveObjects135BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 298 |
| ✓ | BbhLogicalDeductionFiveObjects136BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 824 |
| ✓ | BbhLogicalDeductionFiveObjects137BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 369 |
| ✓ | BbhLogicalDeductionFiveObjects138BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT2 084 |
| ✓ | BbhLogicalDeductionFiveObjects139BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT2 260 |
| ✓ | BbhLogicalDeductionFiveObjects140BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT2 021 |
| ✓ | BbhLogicalDeductionFiveObjects141BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 648 |
| ✓ | BbhLogicalDeductionFiveObjects142BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 296 |
| ✓ | BbhLogicalDeductionFiveObjects143BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT950 |
| ✓ | BbhLogicalDeductionFiveObjects144BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT2 840 |
| ✓ | BbhLogicalDeductionFiveObjects145BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 481 |
| ✓ | BbhLogicalDeductionFiveObjects146BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT844 |
| ✓ | BbhLogicalDeductionFiveObjects147BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 665 |
| ✓ | BbhLogicalDeductionFiveObjects148BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT828 |
| ✓ | BbhLogicalDeductionFiveObjects149BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 336 |
| ✓ | BbhLogicalDeductionFiveObjects150BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 651 |
| ✓ | BbhLogicalDeductionFiveObjects151BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 046 |
| ✓ | BbhLogicalDeductionFiveObjects152BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT1 145 |
| ✓ | BbhLogicalDeductionFiveObjects153BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT1 580 |
| ✓ | BbhLogicalDeductionFiveObjects154BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT2 650 |
| ✓ | BbhLogicalDeductionFiveObjects155BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 534 |
| ✓ | BbhLogicalDeductionFiveObjects156BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT1 466 |
| ✓ | BbhLogicalDeductionFiveObjects157BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 587 |
| ✓ | BbhLogicalDeductionFiveObjects158BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 427 |
Reasoning - BBH logical deduction - order 5 objects (set 8)
30 passed |
100.0% |
IN32 715 OUT46 626 | |
| ✓ | BbhLogicalDeductionFiveObjects159BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 655 |
| ✓ | BbhLogicalDeductionFiveObjects160BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 585 |
| ✓ | BbhLogicalDeductionFiveObjects161BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT1 543 |
| ✓ | BbhLogicalDeductionFiveObjects162BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT972 |
| ✓ | BbhLogicalDeductionFiveObjects163BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 131 |
| ✓ | BbhLogicalDeductionFiveObjects164BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT800 |
| ✓ | BbhLogicalDeductionFiveObjects165BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT2 894 |
| ✓ | BbhLogicalDeductionFiveObjects166BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 237 |
| ✓ | BbhLogicalDeductionFiveObjects167BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 066 |
| ✓ | BbhLogicalDeductionFiveObjects168BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 013 |
| ✓ | BbhLogicalDeductionFiveObjects169BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT2 187 |
| ✓ | BbhLogicalDeductionFiveObjects170BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 691 |
| ✓ | BbhLogicalDeductionFiveObjects171BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT2 070 |
| ✓ | BbhLogicalDeductionFiveObjects172BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 894 |
| ✓ | BbhLogicalDeductionFiveObjects173BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 159 |
| ✓ | BbhLogicalDeductionFiveObjects174BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT449 |
| ✓ | BbhLogicalDeductionFiveObjects175BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 859 |
| ✓ | BbhLogicalDeductionFiveObjects176BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 415 |
| ✓ | BbhLogicalDeductionFiveObjects177BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 879 |
| ✓ | BbhLogicalDeductionFiveObjects178BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 027 |
| ✓ | BbhLogicalDeductionFiveObjects179BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 374 |
| ✓ | BbhLogicalDeductionFiveObjects180BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT2 082 |
| ✓ | BbhLogicalDeductionFiveObjects181BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 631 |
| ✓ | BbhLogicalDeductionFiveObjects182BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 239 |
| ✓ | BbhLogicalDeductionFiveObjects183BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT2 045 |
| ✓ | BbhLogicalDeductionFiveObjects184BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT2 018 |
| ✓ | BbhLogicalDeductionFiveObjects185BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 670 |
| ✓ | BbhLogicalDeductionFiveObjects186BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT972 |
| ✓ | BbhLogicalDeductionFiveObjects187BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 531 |
| ✓ | BbhLogicalDeductionFiveObjects188BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 538 |
Reasoning - BBH logical deduction - order 7 objects (set 1)
32 passed1 failed |
97.0% |
IN37 680 OUT80 566 | |
| ✓ | BbhLogicalDeductionSeven10BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 118 |
| ✓ | BbhLogicalDeductionSeven11BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 046 |
| ✓ | BbhLogicalDeductionSeven12BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 845 |
| ✓ | BbhLogicalDeductionSeven13BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 448 |
| ✓ | BbhLogicalDeductionSeven14BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT2 276 |
| ✓ | BbhLogicalDeductionSeven15BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT2 179 |
| ✓ | BbhLogicalDeductionSeven16BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT4 860 |
| ✓ | BbhLogicalDeductionSeven17BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 888 |
| ✓ | BbhLogicalDeductionSeven18BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT2 168 |
| ✓ | BbhLogicalDeductionSeven19BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 940 |
| ✓ | BbhLogicalDeductionSeven20BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT1 633 |
| ✓ | BbhLogicalDeductionSeven21BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT1 793 |
| ✓ | BbhLogicalDeductionSeven22BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT2 420 |
| ✓ | BbhLogicalDeductionSeven23BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT1 743 |
| ✓ | BbhLogicalDeductionSeven24BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 270 |
| ✓ | BbhLogicalDeductionSeven25BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 852 |
| ✓ | BbhLogicalDeductionSeven26BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT5 239 |
| ✓ | BbhLogicalDeductionSeven27BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT3 075 |
| ✓ | BbhLogicalDeductionSeven28BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT5 281 |
| ✓ | BbhLogicalDeductionSeven29BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT2 907 |
| ✓ | BbhLogicalDeductionSeven30BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 141 OUT2 233 |
| ✓ | BbhLogicalDeductionSeven31BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 082 OUT4 329 |
| ✓ | BbhLogicalDeductionSeven32BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 078 OUT1 486 |
| ✓ | BbhLogicalDeductionSeven33BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 075 OUT1 995 |
| ✓ | BbhLogicalDeductionSeven34BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT2 325 |
| ✓ | BbhLogicalDeductionSeven35BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 902 |
| ✓ | BbhLogicalDeductionSeven36BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT1 614 |
| × | BbhLogicalDeductionSeven37BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhLogicalDeductionSeven37BQ: expected (C), got [(E)] |
Failed | IN1 151 OUT3 397 |
| ✓ | BbhLogicalDeductionSeven38BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 132 OUT1 521 |
| ✓ | BbhLogicalDeductionSeven39BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 098 OUT2 183 |
| ✓ | BbhLogicalDeductionSeven40BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 577 |
| ✓ | BbhLogicalDeductionSeven8BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 134 OUT2 203 |
| ✓ | BbhLogicalDeductionSeven9BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 820 |
Reasoning - BBH logical deduction - order 7 objects (set 2)
33 passed |
100.0% |
IN37 367 OUT70 175 | |
| ✓ | BbhLogicalDeductionSeven41BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT990 |
| ✓ | BbhLogicalDeductionSeven42BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 133 OUT2 269 |
| ✓ | BbhLogicalDeductionSeven43BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT1 884 |
| ✓ | BbhLogicalDeductionSeven44BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 131 OUT3 777 |
| ✓ | BbhLogicalDeductionSeven45BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 224 |
| ✓ | BbhLogicalDeductionSeven46BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT1 688 |
| ✓ | BbhLogicalDeductionSeven47BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 447 |
| ✓ | BbhLogicalDeductionSeven48BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT2 145 |
| ✓ | BbhLogicalDeductionSeven49BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 137 OUT3 283 |
| ✓ | BbhLogicalDeductionSeven50BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 530 |
| ✓ | BbhLogicalDeductionSeven51BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 628 |
| ✓ | BbhLogicalDeductionSeven52BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 091 OUT2 334 |
| ✓ | BbhLogicalDeductionSeven53BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 906 |
| ✓ | BbhLogicalDeductionSeven54BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 078 OUT2 159 |
| ✓ | BbhLogicalDeductionSeven55BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT2 295 |
| ✓ | BbhLogicalDeductionSeven56BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT2 840 |
| ✓ | BbhLogicalDeductionSeven57BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 739 |
| ✓ | BbhLogicalDeductionSeven58BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 545 |
| ✓ | BbhLogicalDeductionSeven59BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT2 532 |
| ✓ | BbhLogicalDeductionSeven60BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 091 OUT2 548 |
| ✓ | BbhLogicalDeductionSeven61BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 503 |
| ✓ | BbhLogicalDeductionSeven62BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 264 |
| ✓ | BbhLogicalDeductionSeven63BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 671 |
| ✓ | BbhLogicalDeductionSeven64BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 082 OUT1 791 |
| ✓ | BbhLogicalDeductionSeven65BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 028 |
| ✓ | BbhLogicalDeductionSeven66BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 561 |
| ✓ | BbhLogicalDeductionSeven67BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 114 |
| ✓ | BbhLogicalDeductionSeven68BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 137 OUT2 288 |
| ✓ | BbhLogicalDeductionSeven69BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 708 |
| ✓ | BbhLogicalDeductionSeven70BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT2 435 |
| ✓ | BbhLogicalDeductionSeven71BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT2 045 |
| ✓ | BbhLogicalDeductionSeven72BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 133 OUT1 852 |
| ✓ | BbhLogicalDeductionSeven73BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 141 OUT2 152 |
Reasoning - BBH logical deduction - order 7 objects (set 3)
31 passed |
100.0% |
IN35 350 OUT71 026 | |
| ✓ | BbhLogicalDeductionSeven100BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 137 OUT1 985 |
| ✓ | BbhLogicalDeductionSeven101BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT1 940 |
| ✓ | BbhLogicalDeductionSeven102BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 851 |
| ✓ | BbhLogicalDeductionSeven103BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT1 380 |
| ✓ | BbhLogicalDeductionSeven104BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT2 386 |
| ✓ | BbhLogicalDeductionSeven74BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 142 OUT1 656 |
| ✓ | BbhLogicalDeductionSeven75BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 803 |
| ✓ | BbhLogicalDeductionSeven76BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 516 |
| ✓ | BbhLogicalDeductionSeven77BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 090 |
| ✓ | BbhLogicalDeductionSeven78BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT2 999 |
| ✓ | BbhLogicalDeductionSeven79BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 225 |
| ✓ | BbhLogicalDeductionSeven80BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT3 530 |
| ✓ | BbhLogicalDeductionSeven81BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 181 OUT3 137 |
| ✓ | BbhLogicalDeductionSeven82BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT3 035 |
| ✓ | BbhLogicalDeductionSeven83BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 136 OUT2 272 |
| ✓ | BbhLogicalDeductionSeven84BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT2 386 |
| ✓ | BbhLogicalDeductionSeven85BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 141 OUT2 094 |
| ✓ | BbhLogicalDeductionSeven86BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 920 |
| ✓ | BbhLogicalDeductionSeven87BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 141 OUT2 271 |
| ✓ | BbhLogicalDeductionSeven88BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT1 939 |
| ✓ | BbhLogicalDeductionSeven89BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 133 OUT1 744 |
| ✓ | BbhLogicalDeductionSeven90BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 078 OUT4 105 |
| ✓ | BbhLogicalDeductionSeven91BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 098 |
| ✓ | BbhLogicalDeductionSeven92BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 112 |
| ✓ | BbhLogicalDeductionSeven93BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT2 047 |
| ✓ | BbhLogicalDeductionSeven94BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT3 061 |
| ✓ | BbhLogicalDeductionSeven95BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT2 610 |
| ✓ | BbhLogicalDeductionSeven96BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 945 |
| ✓ | BbhLogicalDeductionSeven97BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT2 085 |
| ✓ | BbhLogicalDeductionSeven98BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT3 270 |
| ✓ | BbhLogicalDeductionSeven99BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT2 534 |
Reasoning - BBH logical deduction - order 7 objects (set 4)
32 passed |
100.0% |
IN36 292 OUT65 985 | |
| ✓ | BbhLogicalDeductionSeven105BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 996 |
| ✓ | BbhLogicalDeductionSeven106BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 601 |
| ✓ | BbhLogicalDeductionSeven107BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT1 669 |
| ✓ | BbhLogicalDeductionSeven108BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 078 OUT1 787 |
| ✓ | BbhLogicalDeductionSeven109BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 714 |
| ✓ | BbhLogicalDeductionSeven110BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT2 567 |
| ✓ | BbhLogicalDeductionSeven111BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT1 624 |
| ✓ | BbhLogicalDeductionSeven112BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 099 OUT3 040 |
| ✓ | BbhLogicalDeductionSeven113BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT1 706 |
| ✓ | BbhLogicalDeductionSeven114BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 899 |
| ✓ | BbhLogicalDeductionSeven115BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT2 394 |
| ✓ | BbhLogicalDeductionSeven116BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 170 |
| ✓ | BbhLogicalDeductionSeven117BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 535 |
| ✓ | BbhLogicalDeductionSeven118BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT1 662 |
| ✓ | BbhLogicalDeductionSeven119BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 877 |
| ✓ | BbhLogicalDeductionSeven120BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT1 339 |
| ✓ | BbhLogicalDeductionSeven121BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 096 OUT2 608 |
| ✓ | BbhLogicalDeductionSeven122BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT2 143 |
| ✓ | BbhLogicalDeductionSeven123BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 092 OUT1 549 |
| ✓ | BbhLogicalDeductionSeven124BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 078 OUT2 720 |
| ✓ | BbhLogicalDeductionSeven125BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT1 868 |
| ✓ | BbhLogicalDeductionSeven126BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT990 |
| ✓ | BbhLogicalDeductionSeven127BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 081 |
| ✓ | BbhLogicalDeductionSeven128BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 657 |
| ✓ | BbhLogicalDeductionSeven129BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT2 504 |
| ✓ | BbhLogicalDeductionSeven130BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 088 OUT2 965 |
| ✓ | BbhLogicalDeductionSeven131BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT2 057 |
| ✓ | BbhLogicalDeductionSeven132BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 553 |
| ✓ | BbhLogicalDeductionSeven133BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT2 369 |
| ✓ | BbhLogicalDeductionSeven134BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 683 |
| ✓ | BbhLogicalDeductionSeven135BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 134 OUT1 967 |
| ✓ | BbhLogicalDeductionSeven136BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT3 691 |
Reasoning - BBH logical deduction - order 7 objects (set 5)
32 passed |
100.0% |
IN36 403 OUT65 870 | |
| ✓ | BbhLogicalDeductionSeven137BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 967 |
| ✓ | BbhLogicalDeductionSeven138BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT2 408 |
| ✓ | BbhLogicalDeductionSeven139BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 492 |
| ✓ | BbhLogicalDeductionSeven140BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 268 |
| ✓ | BbhLogicalDeductionSeven141BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 855 |
| ✓ | BbhLogicalDeductionSeven142BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 715 |
| ✓ | BbhLogicalDeductionSeven143BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT3 713 |
| ✓ | BbhLogicalDeductionSeven144BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 862 |
| ✓ | BbhLogicalDeductionSeven145BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT2 278 |
| ✓ | BbhLogicalDeductionSeven146BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 913 |
| ✓ | BbhLogicalDeductionSeven147BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT2 348 |
| ✓ | BbhLogicalDeductionSeven148BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 397 |
| ✓ | BbhLogicalDeductionSeven149BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT2 373 |
| ✓ | BbhLogicalDeductionSeven150BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 991 |
| ✓ | BbhLogicalDeductionSeven151BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 145 OUT1 896 |
| ✓ | BbhLogicalDeductionSeven152BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT2 113 |
| ✓ | BbhLogicalDeductionSeven153BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 092 OUT2 552 |
| ✓ | BbhLogicalDeductionSeven154BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 661 |
| ✓ | BbhLogicalDeductionSeven155BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 154 OUT1 927 |
| ✓ | BbhLogicalDeductionSeven156BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 737 |
| ✓ | BbhLogicalDeductionSeven157BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 082 OUT2 630 |
| ✓ | BbhLogicalDeductionSeven158BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT3 144 |
| ✓ | BbhLogicalDeductionSeven159BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 745 |
| ✓ | BbhLogicalDeductionSeven160BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 866 |
| ✓ | BbhLogicalDeductionSeven161BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 088 OUT1 440 |
| ✓ | BbhLogicalDeductionSeven162BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT2 471 |
| ✓ | BbhLogicalDeductionSeven163BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT2 009 |
| ✓ | BbhLogicalDeductionSeven164BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 840 |
| ✓ | BbhLogicalDeductionSeven165BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 543 |
| ✓ | BbhLogicalDeductionSeven166BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 785 |
| ✓ | BbhLogicalDeductionSeven167BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 089 OUT2 576 |
| ✓ | BbhLogicalDeductionSeven168BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 355 |
Reasoning - BBH logical deduction - order 7 objects (set 6)
31 passed1 failed |
96.9% |
IN36 240 OUT69 910 | |
| ✓ | BbhLogicalDeductionSeven169BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT2 160 |
| ✓ | BbhLogicalDeductionSeven170BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT1 755 |
| ✓ | BbhLogicalDeductionSeven171BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 955 |
| ✓ | BbhLogicalDeductionSeven172BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 085 OUT3 402 |
| ✓ | BbhLogicalDeductionSeven173BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT4 197 |
| ✓ | BbhLogicalDeductionSeven174BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT1 925 |
| ✓ | BbhLogicalDeductionSeven175BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 063 |
| ✓ | BbhLogicalDeductionSeven176BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT2 589 |
| ✓ | BbhLogicalDeductionSeven177BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT2 457 |
| ✓ | BbhLogicalDeductionSeven178BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 294 |
| ✓ | BbhLogicalDeductionSeven179BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 222 |
| ✓ | BbhLogicalDeductionSeven180BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 145 OUT1 623 |
| ✓ | BbhLogicalDeductionSeven181BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 093 OUT2 162 |
| ✓ | BbhLogicalDeductionSeven182BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT2 320 |
| ✓ | BbhLogicalDeductionSeven183BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 075 OUT1 224 |
| ✓ | BbhLogicalDeductionSeven184BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT2 622 |
| × | BbhLogicalDeductionSeven185BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhLogicalDeductionSeven185BR: expected (A), got [f]
Values:
got = "f" |
Failed | IN1 155 OUT3 903 |
| ✓ | BbhLogicalDeductionSeven186BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 095 OUT1 972 |
| ✓ | BbhLogicalDeductionSeven187BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 031 |
| ✓ | BbhLogicalDeductionSeven188BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 095 OUT1 331 |
| ✓ | BbhLogicalDeductionSeven189BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 175 OUT1 195 |
| ✓ | BbhLogicalDeductionSeven190BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT3 079 |
| ✓ | BbhLogicalDeductionSeven191BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 141 OUT1 629 |
| ✓ | BbhLogicalDeductionSeven192BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 788 |
| ✓ | BbhLogicalDeductionSeven193BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT3 104 |
| ✓ | BbhLogicalDeductionSeven194BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 698 |
| ✓ | BbhLogicalDeductionSeven195BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 141 OUT1 907 |
| ✓ | BbhLogicalDeductionSeven196BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT1 873 |
| ✓ | BbhLogicalDeductionSeven197BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 132 OUT2 525 |
| ✓ | BbhLogicalDeductionSeven198BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 804 |
| ✓ | BbhLogicalDeductionSeven199BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 366 |
| ✓ | BbhLogicalDeductionSeven200BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 122 OUT1 735 |
Reasoning - BBH logical deduction - order 7 objects (set 7)
32 passed |
100.0% |
IN36 365 OUT81 335 | |
| ✓ | BbhLogicalDeductionSeven201BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 141 OUT3 158 |
| ✓ | BbhLogicalDeductionSeven202BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 586 |
| ✓ | BbhLogicalDeductionSeven203BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 142 OUT2 338 |
| ✓ | BbhLogicalDeductionSeven204BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT2 194 |
| ✓ | BbhLogicalDeductionSeven205BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT2 313 |
| ✓ | BbhLogicalDeductionSeven206BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 138 OUT2 352 |
| ✓ | BbhLogicalDeductionSeven207BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT3 101 |
| ✓ | BbhLogicalDeductionSeven208BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 091 OUT1 400 |
| ✓ | BbhLogicalDeductionSeven209BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 092 OUT1 795 |
| ✓ | BbhLogicalDeductionSeven210BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT2 417 |
| ✓ | BbhLogicalDeductionSeven211BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT2 067 |
| ✓ | BbhLogicalDeductionSeven212BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT4 550 |
| ✓ | BbhLogicalDeductionSeven213BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT3 294 |
| ✓ | BbhLogicalDeductionSeven214BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT2 070 |
| ✓ | BbhLogicalDeductionSeven215BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 137 OUT2 206 |
| ✓ | BbhLogicalDeductionSeven216BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 145 OUT2 380 |
| ✓ | BbhLogicalDeductionSeven217BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 122 OUT2 113 |
| ✓ | BbhLogicalDeductionSeven218BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 137 OUT1 770 |
| ✓ | BbhLogicalDeductionSeven219BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT4 481 |
| ✓ | BbhLogicalDeductionSeven220BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 838 |
| ✓ | BbhLogicalDeductionSeven221BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 088 OUT3 783 |
| ✓ | BbhLogicalDeductionSeven222BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 137 OUT2 861 |
| ✓ | BbhLogicalDeductionSeven223BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT2 615 |
| ✓ | BbhLogicalDeductionSeven224BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 095 OUT1 623 |
| ✓ | BbhLogicalDeductionSeven225BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 181 OUT2 518 |
| ✓ | BbhLogicalDeductionSeven226BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 447 |
| ✓ | BbhLogicalDeductionSeven227BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT4 330 |
| ✓ | BbhLogicalDeductionSeven228BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT2 327 |
| ✓ | BbhLogicalDeductionSeven229BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 106 |
| ✓ | BbhLogicalDeductionSeven230BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 140 OUT2 472 |
| ✓ | BbhLogicalDeductionSeven231BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT2 465 |
| ✓ | BbhLogicalDeductionSeven232BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 365 |
Reasoning - BBH logical deduction - order 7 objects (set 8)
30 passed |
100.0% |
IN33 128 OUT52 919 | |
| ✓ | BbhLogicalDeductionSeven233BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 845 |
| ✓ | BbhLogicalDeductionSeven234BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 132 OUT1 836 |
| ✓ | BbhLogicalDeductionSeven235BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 078 OUT1 901 |
| ✓ | BbhLogicalDeductionSeven236BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 154 OUT2 361 |
| ✓ | BbhLogicalDeductionSeven237BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 912 |
| ✓ | BbhLogicalDeductionSeven238BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 600 |
| ✓ | BbhLogicalDeductionSeven239BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT2 646 |
| ✓ | BbhLogicalDeductionSeven240BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT4 869 |
| ✓ | BbhLogicalDeductionSeven241BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT1 725 |
| ✓ | BbhLogicalDeductionSeven242BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 175 OUT1 442 |
| ✓ | BbhLogicalDeductionSeven243BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT3 253 |
| ✓ | BbhLogicalDeductionSeven244BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT2 383 |
| ✓ | BbhLogicalDeductionSeven245BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 095 OUT1 864 |
| ✓ | BbhLogicalDeductionSeven246BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT2 230 |
| ✓ | BbhLogicalDeductionSeven247BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 682 |
| ✓ | BbhLogicalDeductionSeven248BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 293 |
| ✓ | BbhLogicalDeductionSeven249BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 136 OUT2 701 |
| ✓ | BbhLogicalDeductionThree10BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT440 |
| ✓ | BbhLogicalDeductionThree11BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT885 |
| ✓ | BbhLogicalDeductionThree12BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 252 |
| ✓ | BbhLogicalDeductionThree13BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 128 |
| ✓ | BbhLogicalDeductionThree14BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 224 |
| ✓ | BbhLogicalDeductionThree15BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT975 |
| ✓ | BbhLogicalDeductionThree16BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT473 |
| ✓ | BbhLogicalDeductionThree17BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT890 |
| ✓ | BbhLogicalDeductionThree18BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT386 |
| ✓ | BbhLogicalDeductionThree19BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT644 |
| ✓ | BbhLogicalDeductionThree20BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT717 |
| ✓ | BbhLogicalDeductionThree8BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT2 713 |
| ✓ | BbhLogicalDeductionThree9BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT649 |
Reasoning - BBH movie recommendation (set 1)
30 passed2 failed |
93.8% |
IN32 509 OUT51 690 | |
| ✓ | BbhLogicalDeductionThreeObjects243BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT970 |
| ✓ | BbhLogicalDeductionThreeObjects244BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT966 |
| ✓ | BbhLogicalDeductionThreeObjects245BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT999 |
| ✓ | BbhLogicalDeductionThreeObjects246BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT737 |
| ✓ | BbhLogicalDeductionThreeObjects247BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT1 009 |
| ✓ | BbhLogicalDeductionThreeObjects248BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT2 482 |
| ✓ | BbhLogicalDeductionThreeObjects249BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT457 |
| ✓ | BbhMovieRecommendation10BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT2 454 |
| ✓ | BbhMovieRecommendation11BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 656 |
| ✓ | BbhMovieRecommendation12BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 619 |
| ✓ | BbhMovieRecommendation13BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 417 |
| ✓ | BbhMovieRecommendation14BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 115 |
| ✓ | BbhMovieRecommendation15BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 876 |
| ✓ | BbhMovieRecommendation16BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 344 |
| ✓ | BbhMovieRecommendation17BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT2 017 |
| ✓ | BbhMovieRecommendation18BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 229 |
| ✓ | BbhMovieRecommendation19BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT2 811 |
| ✓ | BbhMovieRecommendation20BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 198 |
| ✓ | BbhMovieRecommendation21BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT2 700 |
| ✓ | BbhMovieRecommendation22BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT1 431 |
| ✓ | BbhMovieRecommendation23BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT2 090 |
| ✓ | BbhMovieRecommendation24BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT2 539 |
| × | BbhMovieRecommendation25BTFind a movie similar to Army of Darkness, Go, The Sixth Sense, Alien:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhMovieRecommendation25BT: expected (A), got [(B)] |
Failed | IN998 OUT1 765 |
| ✓ | BbhMovieRecommendation26BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 658 |
| ✓ | BbhMovieRecommendation27BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 123 |
| ✓ | BbhMovieRecommendation28BTFind a movie similar to Toy Story, The Mask, Batman, The Lion King:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 159 |
| ✓ | BbhMovieRecommendation29BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 454 |
| ✓ | BbhMovieRecommendation30BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT2 363 |
| ✓ | BbhMovieRecommendation31BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 945 |
| ✓ | BbhMovieRecommendation32BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 282 |
| ✓ | BbhMovieRecommendation8BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 414 |
| × | BbhMovieRecommendation9BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhMovieRecommendation9BT: expected (C), got [(B)] |
Failed | IN1 021 OUT1 411 |
Reasoning - BBH movie recommendation (set 2)
29 passed3 failed |
90.6% |
IN32 260 OUT65 335 | |
| ✓ | BbhMovieRecommendation33BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT2 100 |
| ✓ | BbhMovieRecommendation34BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT2 367 |
| ✓ | BbhMovieRecommendation35BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT2 570 |
| ✓ | BbhMovieRecommendation36BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 250 |
| ✓ | BbhMovieRecommendation37BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT6 211 |
| ✓ | BbhMovieRecommendation38BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 164 |
| ✓ | BbhMovieRecommendation39BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT2 961 |
| ✓ | BbhMovieRecommendation40BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 774 |
| ✓ | BbhMovieRecommendation41BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT786 |
| ✓ | BbhMovieRecommendation42BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 634 |
| ✓ | BbhMovieRecommendation43BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT2 094 |
| ✓ | BbhMovieRecommendation44BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 644 |
| ✓ | BbhMovieRecommendation45BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 354 |
| ✓ | BbhMovieRecommendation46BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 360 |
| ✓ | BbhMovieRecommendation47BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 386 |
| ✓ | BbhMovieRecommendation48BTFind a movie similar to Batman, The Mask, The Lion King, Pretty Woman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT2 416 |
| ✓ | BbhMovieRecommendation49BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 878 |
| ✓ | BbhMovieRecommendation50BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT2 435 |
| ✓ | BbhMovieRecommendation51BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 738 |
| × | BbhMovieRecommendation52BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhMovieRecommendation52BT: expected (A), got [(B)] |
Failed | IN1 000 OUT1 674 |
| ✓ | BbhMovieRecommendation53BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 414 |
| ✓ | BbhMovieRecommendation54BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 759 |
| ✓ | BbhMovieRecommendation55BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 606 |
| ✓ | BbhMovieRecommendation56BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT1 201 |
| ✓ | BbhMovieRecommendation57BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT1 575 |
| ✓ | BbhMovieRecommendation58BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 328 |
| ✓ | BbhMovieRecommendation59BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 229 |
| ✓ | BbhMovieRecommendation60BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 298 |
| ✓ | BbhMovieRecommendation61BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 936 |
| × | BbhMovieRecommendation62BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(B)")
Context: BbhMovieRecommendation62BT: expected (B), got [(C)] |
Failed | IN1 006 OUT2 959 |
| ✓ | BbhMovieRecommendation63BTFind a movie similar to Pretty Woman, Aladdin, The Lion King, Batman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT2 415 |
| × | BbhMovieRecommendation64BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(D)")
Context: BbhMovieRecommendation64BT: expected (D), got [(B)] |
Failed | IN1 007 OUT5 819 |
Reasoning - BBH movie recommendation (set 3)
29 passed1 failed |
96.7% |
IN30 288 OUT60 707 | |
| ✓ | BbhMovieRecommendation65BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 268 |
| ✓ | BbhMovieRecommendation66BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT5 323 |
| ✓ | BbhMovieRecommendation67BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 835 |
| ✓ | BbhMovieRecommendation68BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 956 |
| ✓ | BbhMovieRecommendation69BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 685 |
| ✓ | BbhMovieRecommendation70BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 841 |
| ✓ | BbhMovieRecommendation71BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 727 |
| ✓ | BbhMovieRecommendation72BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT2 174 |
| ✓ | BbhMovieRecommendation73BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 492 |
| ✓ | BbhMovieRecommendation74BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT3 538 |
| ✓ | BbhMovieRecommendation75BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 190 |
| ✓ | BbhMovieRecommendation76BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 695 |
| ✓ | BbhMovieRecommendation77BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT2 232 |
| ✓ | BbhMovieRecommendation78BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 799 |
| ✓ | BbhMovieRecommendation79BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT2 986 |
| ✓ | BbhMovieRecommendation80BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT2 749 |
| ✓ | BbhMovieRecommendation81BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT871 |
| ✓ | BbhMovieRecommendation82BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 722 |
| ✓ | BbhMovieRecommendation83BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 336 |
| ✓ | BbhMovieRecommendation84BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT2 160 |
| ✓ | BbhMovieRecommendation85BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 895 |
| ✓ | BbhMovieRecommendation86BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 895 |
| ✓ | BbhMovieRecommendation87BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 397 |
| ✓ | BbhMovieRecommendation88BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT1 969 |
| ✓ | BbhMovieRecommendation89BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 341 |
| ✓ | BbhMovieRecommendation90BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 706 |
| ✓ | BbhMovieRecommendation91BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 958 |
| ✓ | BbhMovieRecommendation92BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT2 768 |
| × | BbhMovieRecommendation93BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(E)")
Context: BbhMovieRecommendation93BT: expected (E), got [(D)] |
Failed | IN1 016 OUT1 476 |
| ✓ | BbhMovieRecommendation94BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 723 |
Reasoning - BBH movie recommendation (set 4)
31 passed1 failed |
96.9% |
IN32 232 OUT76 188 | |
| ✓ | BbhMovieRecommendation100BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT2 460 |
| ✓ | BbhMovieRecommendation101BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 276 |
| ✓ | BbhMovieRecommendation102BUFind a movie similar to The Mask, Batman, Pretty Woman, The Fugitive:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 696 |
| ✓ | BbhMovieRecommendation103BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 395 |
| ✓ | BbhMovieRecommendation104BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT2 037 |
| ✓ | BbhMovieRecommendation105BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT8 914 |
| ✓ | BbhMovieRecommendation106BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 953 |
| ✓ | BbhMovieRecommendation107BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 765 |
| ✓ | BbhMovieRecommendation108BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT3 516 |
| ✓ | BbhMovieRecommendation109BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT2 665 |
| ✓ | BbhMovieRecommendation110BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT3 286 |
| ✓ | BbhMovieRecommendation111BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT3 404 |
| ✓ | BbhMovieRecommendation112BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 326 |
| ✓ | BbhMovieRecommendation113BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT3 138 |
| ✓ | BbhMovieRecommendation114BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 802 |
| ✓ | BbhMovieRecommendation115BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 933 |
| × | BbhMovieRecommendation116BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation116BU: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 018 OUT2 444 |
| ✓ | BbhMovieRecommendation117BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 672 |
| ✓ | BbhMovieRecommendation118BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT3 026 |
| ✓ | BbhMovieRecommendation119BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT2 334 |
| ✓ | BbhMovieRecommendation120BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT2 366 |
| ✓ | BbhMovieRecommendation121BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 334 |
| ✓ | BbhMovieRecommendation122BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 381 |
| ✓ | BbhMovieRecommendation123BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 577 |
| ✓ | BbhMovieRecommendation124BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT1 806 |
| ✓ | BbhMovieRecommendation125BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 041 |
| ✓ | BbhMovieRecommendation126BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 251 |
| ✓ | BbhMovieRecommendation95BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 721 |
| ✓ | BbhMovieRecommendation96BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT2 794 |
| ✓ | BbhMovieRecommendation97BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 275 |
| ✓ | BbhMovieRecommendation98BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT2 406 |
| ✓ | BbhMovieRecommendation99BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT3 194 |
Reasoning - BBH movie recommendation (set 5)
30 passed2 failed |
93.8% |
IN32 260 OUT69 822 | |
| ✓ | BbhMovieRecommendation127BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 417 |
| ✓ | BbhMovieRecommendation128BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 465 |
| ✓ | BbhMovieRecommendation129BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT2 028 |
| ✓ | BbhMovieRecommendation130BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 340 |
| ✓ | BbhMovieRecommendation131BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT2 715 |
| × | BbhMovieRecommendation132BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation132BU: expected (D), got [a]
Values:
got = "a" |
Failed | IN1 019 OUT2 365 |
| ✓ | BbhMovieRecommendation133BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT2 658 |
| ✓ | BbhMovieRecommendation134BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 576 |
| ✓ | BbhMovieRecommendation135BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT8 486 |
| ✓ | BbhMovieRecommendation136BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT2 349 |
| ✓ | BbhMovieRecommendation137BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 045 |
| ✓ | BbhMovieRecommendation138BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 157 |
| ✓ | BbhMovieRecommendation139BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT3 669 |
| ✓ | BbhMovieRecommendation140BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT2 329 |
| ✓ | BbhMovieRecommendation141BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT2 687 |
| ✓ | BbhMovieRecommendation142BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 414 |
| ✓ | BbhMovieRecommendation143BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 867 |
| ✓ | BbhMovieRecommendation144BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT2 637 |
| ✓ | BbhMovieRecommendation145BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT2 769 |
| ✓ | BbhMovieRecommendation146BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 522 |
| ✓ | BbhMovieRecommendation147BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT3 725 |
| ✓ | BbhMovieRecommendation148BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 161 |
| ✓ | BbhMovieRecommendation149BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT669 |
| ✓ | BbhMovieRecommendation150BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 205 |
| ✓ | BbhMovieRecommendation151BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 002 |
| ✓ | BbhMovieRecommendation152BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 262 |
| ✓ | BbhMovieRecommendation153BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT2 121 |
| ✓ | BbhMovieRecommendation154BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT2 163 |
| × | BbhMovieRecommendation155BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation155BU: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 007 OUT1 384 |
| ✓ | BbhMovieRecommendation156BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 683 |
| ✓ | BbhMovieRecommendation157BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 422 |
| ✓ | BbhMovieRecommendation158BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT2 530 |
Reasoning - BBH movie recommendation (set 6)
28 passed4 failed |
87.5% |
IN32 200 OUT67 036 | |
| ✓ | BbhMovieRecommendation159BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 259 |
| ✓ | BbhMovieRecommendation160BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 896 |
| ✓ | BbhMovieRecommendation161BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 540 |
| ✓ | BbhMovieRecommendation162BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 857 |
| × | BbhMovieRecommendation163BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("Monsters, Inc")
Context: BbhMovieRecommendation163BU: expected Monsters, Inc, got [a]
Values:
got = "a" |
Failed | IN1 005 OUT2 425 |
| ✓ | BbhMovieRecommendation164BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 948 |
| ✓ | BbhMovieRecommendation165BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 577 |
| × | BbhMovieRecommendation166BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation166BU: expected (D), got [a]
Values:
got = "a" |
Failed | IN997 OUT2 793 |
| ✓ | BbhMovieRecommendation167BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT4 141 |
| ✓ | BbhMovieRecommendation168BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT2 042 |
| × | BbhMovieRecommendation169BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhMovieRecommendation169BU: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 007 OUT2 885 |
| ✓ | BbhMovieRecommendation170BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 417 |
| ✓ | BbhMovieRecommendation171BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 515 |
| ✓ | BbhMovieRecommendation172BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 426 |
| × | BbhMovieRecommendation173BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation173BU: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 006 OUT3 509 |
| ✓ | BbhMovieRecommendation174BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 381 |
| ✓ | BbhMovieRecommendation175BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT2 022 |
| ✓ | BbhMovieRecommendation176BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 110 |
| ✓ | BbhMovieRecommendation177BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT2 344 |
| ✓ | BbhMovieRecommendation178BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 599 |
| ✓ | BbhMovieRecommendation179BUFind a movie similar to American Beauty, Pi, X-Men, The Godfather:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 567 |
| ✓ | BbhMovieRecommendation180BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT1 196 |
| ✓ | BbhMovieRecommendation181BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT2 283 |
| ✓ | BbhMovieRecommendation182BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT2 048 |
| ✓ | BbhMovieRecommendation183BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 027 |
| ✓ | BbhMovieRecommendation184BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 762 |
| ✓ | BbhMovieRecommendation185BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT2 830 |
| ✓ | BbhMovieRecommendation186BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 712 |
| ✓ | BbhMovieRecommendation187BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 920 |
| ✓ | BbhMovieRecommendation188BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT3 991 |
| ✓ | BbhMovieRecommendation189BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 815 |
| ✓ | BbhMovieRecommendation190BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 199 |
Reasoning - BBH movie recommendation (set 7)
30 passed2 failed |
93.8% |
IN32 272 OUT61 148 | |
| ✓ | BbhMovieRecommendation191BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 162 |
| ✓ | BbhMovieRecommendation192BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 760 |
| ✓ | BbhMovieRecommendation193BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 519 |
| ✓ | BbhMovieRecommendation194BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 717 |
| × | BbhMovieRecommendation195BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhMovieRecommendation195BU: expected (A), got [d]
Values:
got = "d" |
Failed | IN996 OUT2 064 |
| ✓ | BbhMovieRecommendation196BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 121 |
| ✓ | BbhMovieRecommendation197BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 424 |
| ✓ | BbhMovieRecommendation198BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 768 |
| ✓ | BbhMovieRecommendation199BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 310 |
| ✓ | BbhMovieRecommendation200BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT2 509 |
| ✓ | BbhMovieRecommendation201BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT969 |
| ✓ | BbhMovieRecommendation202BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 398 |
| ✓ | BbhMovieRecommendation203BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 572 |
| ✓ | BbhMovieRecommendation204BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 863 |
| ✓ | BbhMovieRecommendation205BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 909 |
| ✓ | BbhMovieRecommendation206BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 891 |
| ✓ | BbhMovieRecommendation207BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT2 407 |
| ✓ | BbhMovieRecommendation208BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 597 |
| ✓ | BbhMovieRecommendation209BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 283 |
| ✓ | BbhMovieRecommendation210BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT4 407 |
| ✓ | BbhMovieRecommendation211BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT2 722 |
| ✓ | BbhMovieRecommendation212BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 322 |
| ✓ | BbhMovieRecommendation213BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 332 |
| ✓ | BbhMovieRecommendation214BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 618 |
| ✓ | BbhMovieRecommendation215BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 852 |
| ✓ | BbhMovieRecommendation216BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 368 |
| ✓ | BbhMovieRecommendation217BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT3 537 |
| ✓ | BbhMovieRecommendation218BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT2 147 |
| ✓ | BbhMovieRecommendation219BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT3 753 |
| ✓ | BbhMovieRecommendation220BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 595 |
| ✓ | BbhMovieRecommendation221BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 804 |
| × | BbhMovieRecommendation222BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation222BU: expected (C), got [b]
Values:
got = "b" |
Failed | IN1 004 OUT2 448 |
Reasoning - BBH movie recommendation (set 8)
27 passed3 failed |
90.0% |
IN30 141 OUT54 379 | |
| × | BbhMovieRecommendation223BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhMovieRecommendation223BU: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 005 OUT2 051 |
| ✓ | BbhMovieRecommendation224BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 734 |
| ✓ | BbhMovieRecommendation225BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 008 |
| ✓ | BbhMovieRecommendation226BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT1 347 |
| ✓ | BbhMovieRecommendation227BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 901 |
| ✓ | BbhMovieRecommendation228BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 428 |
| ✓ | BbhMovieRecommendation229BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 775 |
| ✓ | BbhMovieRecommendation230BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT3 178 |
| ✓ | BbhMovieRecommendation231BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 299 |
| ✓ | BbhMovieRecommendation232BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 453 |
| × | BbhMovieRecommendation233BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation233BU: expected (C), got [b]
Values:
got = "b" |
Failed | IN1 009 OUT1 984 |
| ✓ | BbhMovieRecommendation234BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 496 |
| ✓ | BbhMovieRecommendation235BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 527 |
| ✓ | BbhMovieRecommendation236BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT2 116 |
| ✓ | BbhMovieRecommendation237BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 111 |
| ✓ | BbhMovieRecommendation238BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT2 306 |
| ✓ | BbhMovieRecommendation239BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 539 |
| ✓ | BbhMovieRecommendation240BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 651 |
| ✓ | BbhMovieRecommendation241BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 246 |
| ✓ | BbhMovieRecommendation242BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT1 264 |
| ✓ | BbhMovieRecommendation243BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 410 |
| ✓ | BbhMovieRecommendation244BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 346 |
| ✓ | BbhMovieRecommendation245BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 545 |
| ✓ | BbhMovieRecommendation246BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT2 947 |
| × | BbhMovieRecommendation247BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation247BU: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 005 OUT2 005 |
| ✓ | BbhMovieRecommendation248BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 561 |
| ✓ | BbhMovieRecommendation249BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 371 |
| ✓ | BbhMultistepArithmeticTwo10BU((6 - 0 * 5 + -3) * (6 - -7 + -2 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 202 |
| ✓ | BbhMultistepArithmeticTwo8BU((-7 * -9 + 8 * -3) * (5 + -7 - 4 * -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 061 |
| ✓ | BbhMultistepArithmeticTwo9BU((-9 - 1 * 5 * -5) - (6 + -3 - -1 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 517 |
Reasoning - BBH multi-step arithmetic (set 1)
32 passed |
100.0% |
IN31 328 OUT35 970 | |
| ✓ | BbhMultistepArithmeticTwo11BV((2 - -2 + -7 * 8) * (-7 * -8 * 3 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 326 |
| ✓ | BbhMultistepArithmeticTwo12BV((8 - 2 + -2 * 6) * (8 + -6 + -8 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT851 |
| ✓ | BbhMultistepArithmeticTwo13BV((-6 + -9 - -6 + -4) * (-1 - -6 + -4 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 294 |
| ✓ | BbhMultistepArithmeticTwo14BV((-5 - 4 * -8 + 8) * (4 + 3 - 9 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT983 |
| ✓ | BbhMultistepArithmeticTwo15BV((-5 * -7 * -6 + 9) * (-2 - 8 + -5 + 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT911 |
| ✓ | BbhMultistepArithmeticTwo16BV((8 + 9 - 4 - -9) + (8 + 7 - 6 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT683 |
| ✓ | BbhMultistepArithmeticTwo17BV((5 * -1 + -6 * -3) + (-1 + -8 - 5 + 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT808 |
| ✓ | BbhMultistepArithmeticTwo18BV((-6 - 6 + 7 - 7) - (5 + 3 - 9 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 058 |
| ✓ | BbhMultistepArithmeticTwo19BV((7 - 4 + -3 * 4) - (5 + -8 - 6 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 022 |
| ✓ | BbhMultistepArithmeticTwo20BV((-6 * -1 - 2 + -2) + (9 - 4 + -1 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT788 |
| ✓ | BbhMultistepArithmeticTwo21BV((8 * -6 + 6 * 1) - (-3 * 7 * 0 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT844 |
| ✓ | BbhMultistepArithmeticTwo22BV((-6 - 8 - -7 * -2) - (-9 - 5 + 7 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 362 |
| ✓ | BbhMultistepArithmeticTwo23BV((1 - 7 - -8 * 3) + (-7 - -2 + -3 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 529 |
| ✓ | BbhMultistepArithmeticTwo24BV((5 * -8 - -5 * -9) * (2 - -7 * 6 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 751 |
| ✓ | BbhMultistepArithmeticTwo25BV((6 + 1 - 4 - 3) - (-4 * -6 * -3 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT813 |
| ✓ | BbhMultistepArithmeticTwo26BV((-9 - -9 + 0 + -3) + (-2 - -1 - 1 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT662 |
| ✓ | BbhMultistepArithmeticTwo27BV((9 - 3 + 2 + -1) - (5 - -1 - -6 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 757 |
| ✓ | BbhMultistepArithmeticTwo28BV((-8 + 6 * -2 + 4) * (-4 * 5 + 2 - 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 025 |
| ✓ | BbhMultistepArithmeticTwo29BV((3 + -1 * 7 * -6) - (-7 * -1 + -5 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT964 |
| ✓ | BbhMultistepArithmeticTwo30BV((3 - 3 - -3 + -2) + (8 - 2 + -8 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 031 |
| ✓ | BbhMultistepArithmeticTwo31BV((-2 + -9 - -2 * 2) - (6 * 1 * -2 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 180 |
| ✓ | BbhMultistepArithmeticTwo32BV((-7 + -2 * 1 - 6) + (3 + 9 * 7 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT803 |
| ✓ | BbhMultistepArithmeticTwo33BV((4 + 7 * 4 - -5) - (-4 - 1 - -4 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 461 |
| ✓ | BbhMultistepArithmeticTwo34BV((1 - 6 * 0 - -7) * (6 * 0 + 6 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 028 |
| ✓ | BbhMultistepArithmeticTwo35BV((-1 + -7 + -2 - 7) * (6 - 1 - -1 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 178 |
| ✓ | BbhMultistepArithmeticTwo36BV((-5 + 6 - -5 + -6) + (-4 + -6 - 3 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT693 |
| ✓ | BbhMultistepArithmeticTwo37BV((0 - -1 + -1 + -3) - (-3 * 0 * 8 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT870 |
| ✓ | BbhMultistepArithmeticTwo38BV((-8 + -6 * 9 - 7) + (8 + 2 * -9 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 142 |
| ✓ | BbhMultistepArithmeticTwo39BV((-3 * 0 * 4 - -7) + (-9 * -6 - -8 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT983 |
| ✓ | BbhMultistepArithmeticTwo40BV((-5 - 2 + 9 * -1) * (7 - 5 + 9 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 130 |
| ✓ | BbhMultistepArithmeticTwo41BV((5 - 7 - 8 - 6) * (-2 * -8 * -9 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 089 |
| ✓ | BbhMultistepArithmeticTwo42BV((-5 + 1 - 2 - -3) * (-9 * 3 + 9 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT951 |
Reasoning - BBH multi-step arithmetic (set 2)
32 passed |
100.0% |
IN31 328 OUT39 158 | |
| ✓ | BbhMultistepArithmeticTwo43BV((-2 * -8 * 9 + -5) + (7 - -1 - -4 - 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 363 |
| ✓ | BbhMultistepArithmeticTwo44BV((-2 - 5 * -5 - 6) + (-5 * -4 + 2 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 152 |
| ✓ | BbhMultistepArithmeticTwo45BV((-5 - 6 * 5 + -9) + (1 + -4 - -2 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 446 |
| ✓ | BbhMultistepArithmeticTwo46BV((-8 + 0 * 0 * -1) - (7 + -1 + -3 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT984 |
| ✓ | BbhMultistepArithmeticTwo47BV((-9 - -3 * 1 - -3) * (-9 - 5 - -2 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 537 |
| ✓ | BbhMultistepArithmeticTwo48BV((5 * -2 + 8 * 1) * (9 + -3 * -8 - -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 232 |
| ✓ | BbhMultistepArithmeticTwo49BV((-7 * 9 - -9 + 7) + (9 * -6 * -4 * -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 452 |
| ✓ | BbhMultistepArithmeticTwo50BV((4 - 3 * -1 * 3) * (3 + 8 + 1 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 064 |
| ✓ | BbhMultistepArithmeticTwo51BV((-3 - 8 * 5 + 5) - (-1 * -3 - -2 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 403 |
| ✓ | BbhMultistepArithmeticTwo52BV((1 + 3 - -5 - 4) + (-3 - -3 + 6 * 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 134 |
| ✓ | BbhMultistepArithmeticTwo53BV((3 * 9 - 3 * 2) + (-1 * -7 - 1 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT714 |
| ✓ | BbhMultistepArithmeticTwo54BV((7 * 9 * 3 * 6) * (-8 * -6 + -9 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 704 |
| ✓ | BbhMultistepArithmeticTwo55BV((1 + 1 + -7 * -6) - (4 * 2 * -5 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT829 |
| ✓ | BbhMultistepArithmeticTwo56BV((1 + 1 * 3 - 2) * (-8 - -2 - 9 + -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 341 |
| ✓ | BbhMultistepArithmeticTwo57BV((5 + 6 * 4 * 9) + (-9 - 0 * 3 - 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT770 |
| ✓ | BbhMultistepArithmeticTwo58BV((-1 - 9 + -9 * -9) - (1 + -3 - 5 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 504 |
| ✓ | BbhMultistepArithmeticTwo59BV((-1 + -6 - -4 + -6) - (-2 * -5 - -1 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT965 |
| ✓ | BbhMultistepArithmeticTwo60BV((-2 - -7 + 5 * -3) + (2 - 8 + 5 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 418 |
| ✓ | BbhMultistepArithmeticTwo61BV((3 - -6 - 7 + 3) + (-7 + 6 * 9 + 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 049 |
| ✓ | BbhMultistepArithmeticTwo62BV((-1 - -9 * 8 - -9) + (-9 - 2 * -6 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 271 |
| ✓ | BbhMultistepArithmeticTwo63BV((-5 + 6 + -6 * -1) + (-5 + -5 + 9 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT690 |
| ✓ | BbhMultistepArithmeticTwo64BV((4 + -1 + -7 + 2) - (-7 + -9 * -6 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT779 |
| ✓ | BbhMultistepArithmeticTwo65BV((8 * 9 * 0 - -1) - (-9 - -7 + -4 - 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 101 |
| ✓ | BbhMultistepArithmeticTwo66BV((4 + 5 * -9 * -8) - (-9 + -7 - -6 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 244 |
| ✓ | BbhMultistepArithmeticTwo67BV((-8 + 5 - 0 - 6) * (-5 * -4 - 6 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT736 |
| ✓ | BbhMultistepArithmeticTwo68BV((-4 + -5 * 7 + 7) + (3 * -3 - -4 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 041 |
| ✓ | BbhMultistepArithmeticTwo69BV((0 * 8 - -9 * 6) - (-3 * -9 * -1 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 208 |
| ✓ | BbhMultistepArithmeticTwo70BV((0 * 9 + -8 * 7) - (7 - 9 + 1 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 049 |
| ✓ | BbhMultistepArithmeticTwo71BV((9 * 4 - 6 - -8) + (6 - 8 - -4 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 561 |
| ✓ | BbhMultistepArithmeticTwo72BV((6 - 1 - -3 + -2) + (5 * 5 + -1 - -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 142 |
| ✓ | BbhMultistepArithmeticTwo73BV((-4 + 4 + 3 * 1) - (-9 - -3 + -6 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 127 |
| ✓ | BbhMultistepArithmeticTwo74BV((-3 - -6 - -7 + 3) + (5 * 5 * 1 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 148 |
Reasoning - BBH multi-step arithmetic (set 3)
32 passed |
100.0% |
IN31 328 OUT46 832 | |
| ✓ | BbhMultistepArithmeticTwo100BV((7 - 5 * 7 + -1) + (6 * 4 - 9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 233 |
| ✓ | BbhMultistepArithmeticTwo101BV((6 + -8 * -3 - -5) * (2 - 7 * -2 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 856 |
| ✓ | BbhMultistepArithmeticTwo102BV((-1 * -1 * -6 - 7) - (-4 * 0 - -4 - 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT878 |
| ✓ | BbhMultistepArithmeticTwo103BV((-6 + 1 - -4 * -2) + (4 - -8 - -1 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 321 |
| ✓ | BbhMultistepArithmeticTwo104BV((7 + -9 * -5 + -3) * (-8 * 8 - -5 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 221 |
| ✓ | BbhMultistepArithmeticTwo105BV((3 * 4 * 0 * -3) - (6 + -4 + 2 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 099 |
| ✓ | BbhMultistepArithmeticTwo106BV((2 + -6 * -7 - 7) + (-2 - -5 + -4 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 868 |
| ✓ | BbhMultistepArithmeticTwo75BV((-8 * 4 + 7 - 5) + (9 + -6 + -2 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 261 |
| ✓ | BbhMultistepArithmeticTwo76BV((-8 - -6 * 7 * 0) * (0 + -5 * -8 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 676 |
| ✓ | BbhMultistepArithmeticTwo77BV((5 * 1 * 3 - 9) * (-7 + 4 - -7 * -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 674 |
| ✓ | BbhMultistepArithmeticTwo78BV((5 - -2 * -9 + 8) - (3 - 4 * -3 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT4 421 |
| ✓ | BbhMultistepArithmeticTwo79BV((7 + 1 + 9 * 1) + (7 - -3 + 3 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 546 |
| ✓ | BbhMultistepArithmeticTwo80BV((-6 + -2 * 3 * 0) + (3 + -4 - 7 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 454 |
| ✓ | BbhMultistepArithmeticTwo81BV((-8 + -4 * -9 * 0) + (2 * -9 + 3 * -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 280 |
| ✓ | BbhMultistepArithmeticTwo82BV((-8 + 6 - 8 * -9) - (9 - 9 * -2 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 811 |
| ✓ | BbhMultistepArithmeticTwo83BV((9 - -6 * 9 * 4) - (6 - 4 * -8 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 079 |
| ✓ | BbhMultistepArithmeticTwo84BV((8 + 3 * -8 - -8) - (2 + 2 * -9 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 318 |
| ✓ | BbhMultistepArithmeticTwo85BV((-9 + 8 - -4 * 2) - (1 - 9 + -6 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 143 |
| ✓ | BbhMultistepArithmeticTwo86BV((-1 * -6 - -1 - -8) - (8 + -7 * 8 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 242 |
| ✓ | BbhMultistepArithmeticTwo87BV((-3 * 6 - 0 * 6) - (-6 * -6 * 7 + -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT855 |
| ✓ | BbhMultistepArithmeticTwo88BV((-6 - -1 - -5 * -5) + (8 + -5 - 9 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 499 |
| ✓ | BbhMultistepArithmeticTwo89BV((-9 + -4 + 0 * -1) * (4 * -3 * -8 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 216 |
| ✓ | BbhMultistepArithmeticTwo90BV((-2 - 2 * 2 + 0) - (7 - 6 - -7 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 248 |
| ✓ | BbhMultistepArithmeticTwo91BV((-7 - -4 * 2 - 9) * (9 - 6 - 1 + -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 996 |
| ✓ | BbhMultistepArithmeticTwo92BV((-9 * 4 * -9 - -9) * (5 * 2 * 6 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 200 |
| ✓ | BbhMultistepArithmeticTwo93BV((-9 + -3 + 6 - 8) - (-7 * -2 * -6 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 268 |
| ✓ | BbhMultistepArithmeticTwo94BV((-7 + 5 * -6 * -6) * (7 * -5 + -6 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 138 |
| ✓ | BbhMultistepArithmeticTwo95BV((-8 * 0 + 2 * 2) * (-9 - 1 * -9 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 007 |
| ✓ | BbhMultistepArithmeticTwo96BV((9 + 2 + 3 - -3) + (-5 + 6 - -6 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT952 |
| ✓ | BbhMultistepArithmeticTwo97BV((5 - 6 - 7 + 6) - (-1 - 1 * -7 + 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 131 |
| ✓ | BbhMultistepArithmeticTwo98BV((-9 - -7 + 2 + 2) + (-7 + -9 - -1 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 106 |
| ✓ | BbhMultistepArithmeticTwo99BV((-6 * -4 + -1 + 1) + (-9 * -6 + 7 + 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT835 |
Reasoning - BBH multi-step arithmetic (set 4)
31 passed |
100.0% |
IN30 349 OUT39 717 | |
| ✓ | BbhMultistepArithmeticTwo107BV((-3 + -4 * 5 * -7) - (-1 * -4 + -9 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT944 |
| ✓ | BbhMultistepArithmeticTwo108BV((0 + -9 * 6 * 4) + (5 - -3 + 8 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 217 |
| ✓ | BbhMultistepArithmeticTwo109BV((-4 - -6 - -5 - 5) - (-2 + -4 - -8 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 202 |
| ✓ | BbhMultistepArithmeticTwo110BV((-3 + -4 * 3 - -8) + (8 - 1 - -4 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 421 |
| ✓ | BbhMultistepArithmeticTwo111BV((-4 - 5 + 2 + 7) - (-9 * 6 * -1 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 253 |
| ✓ | BbhMultistepArithmeticTwo112BV((8 + 1 + 6 + 6) - (7 - -7 - 9 - -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT746 |
| ✓ | BbhMultistepArithmeticTwo113BV((-9 * -5 - 5 + 7) - (-3 - 8 + 9 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 651 |
| ✓ | BbhMultistepArithmeticTwo114BV((0 + 1 - -6 * 9) * (4 + 7 + -9 - 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 173 |
| ✓ | BbhMultistepArithmeticTwo115BV((-1 + -6 + -3 * -2) * (-9 * -1 + 2 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT809 |
| ✓ | BbhMultistepArithmeticTwo116BV((6 * 5 + 0 - -3) * (4 * 5 * 3 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 041 |
| ✓ | BbhMultistepArithmeticTwo117BV((-2 * -1 + 1 * -3) - (7 * -7 + -5 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT871 |
| ✓ | BbhMultistepArithmeticTwo118BV((-1 + 5 - -1 + 3) + (0 - -3 - -5 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT863 |
| ✓ | BbhMultistepArithmeticTwo119BV((0 - 5 + 0 + 3) * (-7 + -8 * 4 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT951 |
| ✓ | BbhMultistepArithmeticTwo120BV((-2 - 3 - 0 - 2) * (1 - -7 - -3 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT693 |
| ✓ | BbhMultistepArithmeticTwo121BV((3 + 3 - -4 - 2) + (-2 - -7 * -2 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 952 |
| ✓ | BbhMultistepArithmeticTwo122BV((-9 - -8 + 3 * -1) * (-9 * -2 * 4 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 089 |
| ✓ | BbhMultistepArithmeticTwo123BV((-1 - 9 * 6 + 3) * (0 - 8 - -4 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 801 |
| ✓ | BbhMultistepArithmeticTwo124BV((-9 - 8 * 9 - 0) + (-4 + 4 + -2 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT742 |
| ✓ | BbhMultistepArithmeticTwo125BV((-6 - -5 - -9 - -9) * (-6 * 4 - 8 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 188 |
| ✓ | BbhMultistepArithmeticTwo126BV((1 + 2 * -1 - -8) - (-8 - 2 - -1 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 187 |
| ✓ | BbhMultistepArithmeticTwo127BV((-6 * -2 + -2 - -9) + (-9 * 3 * -1 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 177 |
| ✓ | BbhMultistepArithmeticTwo128BV((-1 * -3 - 2 + -7) + (4 * 8 * -6 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 096 |
| ✓ | BbhMultistepArithmeticTwo129BV((8 + 6 * -1 - -3) * (-5 * 0 - -7 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 326 |
| ✓ | BbhMultistepArithmeticTwo130BV((-5 - -1 - 7 * 8) * (-1 - -4 * -7 * 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 179 |
| ✓ | BbhMultistepArithmeticTwo131BV((4 - 8 - -9 * 7) - (5 + -8 * 9 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 160 |
| ✓ | BbhMultistepArithmeticTwo132BV((-7 - -8 + 5 - -4) + (6 - -6 * 4 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 725 |
| ✓ | BbhMultistepArithmeticTwo133BV((-3 * 4 * 3 + -5) + (-7 - 5 + 7 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT910 |
| ✓ | BbhMultistepArithmeticTwo134BV((-2 - 1 - -6 + -9) - (4 * 8 + -8 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT921 |
| ✓ | BbhMultistepArithmeticTwo135BV((-2 * 4 - 3 - 3) - (-9 * -3 - -8 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT3 471 |
| ✓ | BbhMultistepArithmeticTwo136BV((-4 + -6 + -6 - -9) + (-2 * -5 * 5 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT964 |
| ✓ | BbhMultistepArithmeticTwo137BV((-5 * 5 + 1 - 4) + (9 * 0 - -7 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT994 |
Reasoning - BBH multi-step arithmetic (set 5)
31 passed |
100.0% |
IN30 349 OUT38 988 | |
| ✓ | BbhMultistepArithmeticTwo138BV((1 - -8 + 0 - -3) + (-3 + -4 + 1 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 151 |
| ✓ | BbhMultistepArithmeticTwo139BV((8 - -5 + -9 + 7) + (-9 * -1 * -6 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 103 |
| ✓ | BbhMultistepArithmeticTwo140BV((0 * 3 - -9 + -1) - (-5 * -6 - 6 + 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT986 |
| ✓ | BbhMultistepArithmeticTwo141BV((-4 + -9 + -7 * 8) - (-8 + 0 * -5 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 067 |
| ✓ | BbhMultistepArithmeticTwo142BV((7 + -7 * 9 * 5) - (-2 + -6 - -8 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT6 818 |
| ✓ | BbhMultistepArithmeticTwo143BV((-4 - -8 + 2 + -8) * (-3 * 0 + 4 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT857 |
| ✓ | BbhMultistepArithmeticTwo144BV((3 - 9 - 3 - 4) + (1 * -2 * -3 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 192 |
| ✓ | BbhMultistepArithmeticTwo145BV((-6 * -4 + 1 + -1) * (-1 + 2 * 7 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 093 |
| ✓ | BbhMultistepArithmeticTwo146BV((-9 * -3 - 8 - 5) + (1 * -2 - -4 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 112 |
| ✓ | BbhMultistepArithmeticTwo147BV((3 + -6 * -2 + 7) * (-6 - -4 - -6 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 104 |
| ✓ | BbhMultistepArithmeticTwo148BV((-7 * 2 + -4 * -8) - (-5 * 0 * 1 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT953 |
| ✓ | BbhMultistepArithmeticTwo149BV((-4 * 4 + -1 * 5) - (-6 + 1 - -8 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 666 |
| ✓ | BbhMultistepArithmeticTwo150BV((7 - -9 + 0 - 0) + (-9 + 5 - -9 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT979 |
| ✓ | BbhMultistepArithmeticTwo151BV((4 + 3 + -9 + -2) * (7 * 2 + 8 - 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT691 |
| ✓ | BbhMultistepArithmeticTwo152BV((3 - 1 + -3 * -4) - (8 * 9 - -7 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT951 |
| ✓ | BbhMultistepArithmeticTwo153BV((-6 * -7 * -8 - -2) + (-5 - 2 + 0 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 081 |
| ✓ | BbhMultistepArithmeticTwo154BV((5 * 7 * -4 + 7) * (-5 - -6 + -8 * -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 271 |
| ✓ | BbhMultistepArithmeticTwo155BV((4 - 2 * 2 + 3) + (-3 - -9 - -4 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 090 |
| ✓ | BbhMultistepArithmeticTwo156BV((8 - 7 - -3 + -9) - (-6 * -6 + -4 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT716 |
| ✓ | BbhMultistepArithmeticTwo157BV((-7 + 9 + -9 * -4) - (-7 * 6 - 7 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 146 |
| ✓ | BbhMultistepArithmeticTwo158BV((-6 * -2 + 6 - -8) - (-2 * -4 * 8 + 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 229 |
| ✓ | BbhMultistepArithmeticTwo159BV((-1 + 5 * 4 + 9) - (-7 - 6 * -8 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 342 |
| ✓ | BbhMultistepArithmeticTwo160BV((-4 + -2 + -6 + -1) + (-4 * 2 + -5 * 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT674 |
| ✓ | BbhMultistepArithmeticTwo161BV((-8 - -1 - -2 * -7) + (3 + 2 - 3 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 124 |
| ✓ | BbhMultistepArithmeticTwo162BV((4 + 5 - 7 + 9) * (-1 - -4 + -7 - -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT796 |
| ✓ | BbhMultistepArithmeticTwo163BV((7 + 7 * 3 - 3) - (-3 + 4 - -6 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 109 |
| ✓ | BbhMultistepArithmeticTwo164BV((-4 + 5 - 4 - -6) * (-3 * -1 * 2 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT694 |
| ✓ | BbhMultistepArithmeticTwo165BV((2 + 7 - -5 - 1) - (0 - 5 - 9 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT737 |
| ✓ | BbhMultistepArithmeticTwo166BV((4 * -4 + -6 - -2) - (9 + -7 + -8 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 416 |
| ✓ | BbhMultistepArithmeticTwo167BV((3 - 2 - 3 * 3) + (-8 * -1 * -9 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 015 |
| ✓ | BbhMultistepArithmeticTwo168BV((-1 * -5 - -3 + -9) + (-5 - 3 * -2 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT825 |
Reasoning - BBH multi-step arithmetic (set 6)
32 passed |
100.0% |
IN31 328 OUT41 682 | |
| ✓ | BbhMultistepArithmeticTwo169BW((5 * -1 - 7 * 5) - (6 - -5 * 3 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 831 |
| ✓ | BbhMultistepArithmeticTwo170BW((9 - -5 - 1 + -2) * (-5 * 3 - -5 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT830 |
| ✓ | BbhMultistepArithmeticTwo171BW((-8 - -3 * 5 + 4) * (2 * 7 - -3 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 493 |
| ✓ | BbhMultistepArithmeticTwo172BW((5 - 7 + -6 + 4) * (2 - -9 + 3 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT699 |
| ✓ | BbhMultistepArithmeticTwo173BW((-5 - -1 - 6 * 0) + (-3 + 5 * 0 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT882 |
| ✓ | BbhMultistepArithmeticTwo174BW((-3 - 9 - 3 - 1) + (-4 * -9 * -7 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT892 |
| ✓ | BbhMultistepArithmeticTwo175BW((1 * -1 + 5 * -4) + (-2 - 5 * 4 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 028 |
| ✓ | BbhMultistepArithmeticTwo176BW((-3 - -8 - -7 + -2) - (-2 - 8 + -1 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 156 |
| ✓ | BbhMultistepArithmeticTwo177BW((-5 * -6 + 3 * -3) - (-1 + -8 * 8 * -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 294 |
| ✓ | BbhMultistepArithmeticTwo178BW((-5 - -7 - 4 - 8) - (-4 + 4 + 1 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT855 |
| ✓ | BbhMultistepArithmeticTwo179BW((9 + 7 - 4 - 7) - (-5 + 6 + -1 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT972 |
| ✓ | BbhMultistepArithmeticTwo180BW((8 * -6 + -9 + 9) * (0 - 4 - 0 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT925 |
| ✓ | BbhMultistepArithmeticTwo181BW((-4 * 1 + 8 + 9) * (7 * -6 * -5 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 170 |
| ✓ | BbhMultistepArithmeticTwo182BW((-5 - 6 * 2 + 4) - (1 - 9 - 0 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 009 |
| ✓ | BbhMultistepArithmeticTwo183BW((-7 - 1 + 3 * -2) + (0 - 4 + 6 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 033 |
| ✓ | BbhMultistepArithmeticTwo184BW((7 * 2 + 4 - -6) * (6 - 7 - 7 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 022 |
| ✓ | BbhMultistepArithmeticTwo185BW((-6 * -9 + 1 * 7) - (0 + 3 * 5 - 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT990 |
| ✓ | BbhMultistepArithmeticTwo186BW((-8 - -1 + -8 + -3) * (-4 * 6 - 2 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 091 |
| ✓ | BbhMultistepArithmeticTwo187BW((0 * -4 - -7 * 5) + (-4 * 0 + 6 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 439 |
| ✓ | BbhMultistepArithmeticTwo188BW((9 * -9 + -4 * -8) - (5 * 0 + 8 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT931 |
| ✓ | BbhMultistepArithmeticTwo189BW((-6 + -9 - -6 + -1) + (-8 * 5 * -6 + 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT827 |
| ✓ | BbhMultistepArithmeticTwo190BW((5 + 4 + 7 - 6) - (-8 * 0 - 9 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT853 |
| ✓ | BbhMultistepArithmeticTwo191BW((9 - 6 + -4 + -5) * (4 - -8 + -9 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT565 |
| ✓ | BbhMultistepArithmeticTwo192BW((-9 * 6 - -3 * -3) + (-4 - -9 + 4 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT3 533 |
| ✓ | BbhMultistepArithmeticTwo193BW((-8 * -8 - -5 * -6) * (3 + -9 - 5 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 843 |
| ✓ | BbhMultistepArithmeticTwo194BW((0 + 3 * 4 * 9) - (4 - -5 + -4 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 057 |
| ✓ | BbhMultistepArithmeticTwo195BW((-4 + -7 - 3 + 3) - (-3 + -2 - -3 * -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT3 162 |
| ✓ | BbhMultistepArithmeticTwo196BW((-7 * -2 * -1 + -9) - (-7 + 5 - 0 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 154 |
| ✓ | BbhMultistepArithmeticTwo197BW((-3 + 2 * -7 * 5) * (-9 - 9 - -9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 532 |
| ✓ | BbhMultistepArithmeticTwo198BW((1 + 0 + 2 - 4) + (-9 + 6 * -5 + 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT787 |
| ✓ | BbhMultistepArithmeticTwo199BW((-3 + -4 * 4 + -8) + (1 - -3 + 4 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 031 |
| ✓ | BbhMultistepArithmeticTwo200BW((3 * -7 * -4 * 2) + (7 - -3 - -3 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 796 |
Reasoning - BBH multi-step arithmetic (set 7)
32 passed |
100.0% |
IN31 328 OUT40 364 | |
| ✓ | BbhMultistepArithmeticTwo201BW((-5 * -6 + -9 - 3) + (-7 - -2 - 6 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 451 |
| ✓ | BbhMultistepArithmeticTwo202BW((6 - 5 + 5 + -9) - (9 + 8 * -6 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT992 |
| ✓ | BbhMultistepArithmeticTwo203BW((8 * 3 - -1 - -3) * (-4 * 7 * -3 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 341 |
| ✓ | BbhMultistepArithmeticTwo204BW((5 + 3 - 6 + 1) + (-5 + 5 - 6 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT804 |
| ✓ | BbhMultistepArithmeticTwo205BW((-9 + 6 + 1 - 7) * (2 - -2 + -9 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT830 |
| ✓ | BbhMultistepArithmeticTwo206BW((1 * 0 - -5 - 3) + (9 * 6 + -4 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT946 |
| ✓ | BbhMultistepArithmeticTwo207BW((1 - 2 + 5 * 2) + (-2 - -7 + 3 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 035 |
| ✓ | BbhMultistepArithmeticTwo208BW((5 + -7 + 1 - 5) - (-3 * -1 * 7 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT904 |
| ✓ | BbhMultistepArithmeticTwo209BW((-7 * -2 + -4 + -4) - (9 * 1 * -6 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 480 |
| ✓ | BbhMultistepArithmeticTwo210BW((1 + 0 - 3 + 0) - (-7 - 9 * 1 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 045 |
| ✓ | BbhMultistepArithmeticTwo211BW((7 * 6 - -2 - 7) * (4 * 5 * -3 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 301 |
| ✓ | BbhMultistepArithmeticTwo212BW((3 * -9 * 3 - 7) - (-9 + 4 * 8 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 466 |
| ✓ | BbhMultistepArithmeticTwo213BW((9 + 6 - 9 + 8) * (9 - -9 * -6 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 149 |
| ✓ | BbhMultistepArithmeticTwo214BW((5 * 6 + -6 - 8) * (-5 - 6 - 5 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 058 |
| ✓ | BbhMultistepArithmeticTwo215BW((5 + 4 - -1 * -1) + (-6 - -7 + -1 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 407 |
| ✓ | BbhMultistepArithmeticTwo216BW((9 - 5 - 4 + -2) + (-9 + 1 - 1 - 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT773 |
| ✓ | BbhMultistepArithmeticTwo217BW((6 - 9 * -6 - -9) + (3 * 3 + -5 * -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT863 |
| ✓ | BbhMultistepArithmeticTwo218BW((-2 * 5 + -3 + 0) * (-9 + 2 + 5 * 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT708 |
| ✓ | BbhMultistepArithmeticTwo219BW((5 * -3 * 3 * 7) * (0 + -9 - -5 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 153 |
| ✓ | BbhMultistepArithmeticTwo220BW((8 - 3 + 0 + 7) + (-9 * -6 * 2 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT943 |
| ✓ | BbhMultistepArithmeticTwo221BW((-2 * 9 * -8 + 8) - (0 * -4 - -6 * 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 214 |
| ✓ | BbhMultistepArithmeticTwo222BW((-7 * -6 * 0 - 5) * (-3 - 9 * -8 * -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 148 |
| ✓ | BbhMultistepArithmeticTwo223BW((3 * 8 * -1 * -9) - (-2 + 2 * -8 * 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 486 |
| ✓ | BbhMultistepArithmeticTwo224BW((-9 + 5 * 0 - -8) + (6 - -5 * -7 + -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 591 |
| ✓ | BbhMultistepArithmeticTwo225BW((-2 - -7 - 5 * 2) + (-3 - 5 * 3 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT906 |
| ✓ | BbhMultistepArithmeticTwo226BW((-9 + -7 + -1 - -1) * (3 - -7 - -1 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 326 |
| ✓ | BbhMultistepArithmeticTwo227BW((-1 + 6 + 9 + 3) - (-5 - 9 - -3 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT832 |
| ✓ | BbhMultistepArithmeticTwo228BW((-7 + 9 * 9 * 6) - (-2 - -5 + 9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 192 |
| ✓ | BbhMultistepArithmeticTwo229BW((-8 - -7 + 0 + 3) + (-3 + 5 - -5 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT962 |
| ✓ | BbhMultistepArithmeticTwo230BW((8 - 4 - -8 - -7) + (6 - 5 - -1 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 122 |
| ✓ | BbhMultistepArithmeticTwo231BW((-6 * 8 + -6 - -5) + (-4 - 9 * 5 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT905 |
| ✓ | BbhMultistepArithmeticTwo232BW((5 * -9 - 4 - 4) - (-3 * 2 * 9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 031 |
Reasoning - BBH multi-step arithmetic (set 8)
32 passed |
100.0% |
IN31 687 OUT43 055 | |
| ✓ | BbhMultistepArithmeticTwo233BW((4 * -9 * 3 * -8) + (-9 - 4 - -9 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT882 |
| ✓ | BbhMultistepArithmeticTwo234BW((-7 - 2 - 0 + 2) * (-4 * -5 + -8 + 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT915 |
| ✓ | BbhMultistepArithmeticTwo235BW((-1 + 7 * 7 * -9) * (4 + 3 + -8 * 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 409 |
| ✓ | BbhMultistepArithmeticTwo236BW((7 * 8 + 1 + 3) - (3 + -1 - -9 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT972 |
| ✓ | BbhMultistepArithmeticTwo237BW((3 + -9 + 8 + -4) * (-4 + -6 - 4 - -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT761 |
| ✓ | BbhMultistepArithmeticTwo238BW((1 * 5 * -8 - -2) * (-2 * 6 - -5 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 258 |
| ✓ | BbhMultistepArithmeticTwo239BW((-8 - -1 * -8 * -2) * (-6 + 8 + -5 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 374 |
| ✓ | BbhMultistepArithmeticTwo240BW((4 - -6 + -9 + -7) - (9 + -3 - -7 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 240 |
| ✓ | BbhMultistepArithmeticTwo241BW((0 * -7 - 6 * 3) * (-9 + -5 - 2 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT978 |
| ✓ | BbhMultistepArithmeticTwo242BW((-8 * 1 * -1 + -5) + (-2 - -3 + 1 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 194 |
| ✓ | BbhMultistepArithmeticTwo243BW((8 - -9 * 4 * 9) * (-4 * -3 * 7 * 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 572 |
| ✓ | BbhMultistepArithmeticTwo244BW((1 * -9 - 4 + -9) + (-1 + -6 + 9 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT984 |
| ✓ | BbhMultistepArithmeticTwo245BW((-1 + 2 - 9 + 0) * (0 - -2 + -6 - 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT849 |
| ✓ | BbhMultistepArithmeticTwo246BW((7 * -5 * -9 + 7) - (2 + -8 * 6 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT995 |
| ✓ | BbhMultistepArithmeticTwo247BW((-5 + -8 - -6 * 7) + (9 * -7 - -5 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT3 157 |
| ✓ | BbhMultistepArithmeticTwo248BW((-6 * 9 - -5 + 8) * (-8 * -6 - -1 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 266 |
| ✓ | BbhMultistepArithmeticTwo249BW((5 - -2 * 7 + -2) - (0 * 9 * -6 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 090 |
| ✓ | BbhNavigate10BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 578 |
| ✓ | BbhNavigate11BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 924 |
| ✓ | BbhNavigate12BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 331 |
| ✓ | BbhNavigate13BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT2 154 |
| ✓ | BbhNavigate14BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 821 |
| ✓ | BbhNavigate15BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 353 |
| ✓ | BbhNavigate16BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT638 |
| ✓ | BbhNavigate17BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 060 |
| ✓ | BbhNavigate18BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT1 107 |
| ✓ | BbhNavigate19BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT880 |
| ✓ | BbhNavigate20BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT845 |
| ✓ | BbhNavigate21BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT900 |
| ✓ | BbhNavigate22BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT2 024 |
| ✓ | BbhNavigate8BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT910 |
| ✓ | BbhNavigate9BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT3 634 |
Reasoning - BBH navigation - returns to start? (set 1)
32 passed |
100.0% |
IN32 148 OUT46 479 | |
| ✓ | BbhNavigate23BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 415 |
| ✓ | BbhNavigate24BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT597 |
| ✓ | BbhNavigate25BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 592 |
| ✓ | BbhNavigate26BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT440 |
| ✓ | BbhNavigate27BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 390 |
| ✓ | BbhNavigate28BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 169 |
| ✓ | BbhNavigate29BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT936 |
| ✓ | BbhNavigate30BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 421 |
| ✓ | BbhNavigate31BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT4 732 |
| ✓ | BbhNavigate32BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 060 |
| ✓ | BbhNavigate33BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT2 188 |
| ✓ | BbhNavigate34BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 094 |
| ✓ | BbhNavigate35BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT834 |
| ✓ | BbhNavigate36BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT2 447 |
| ✓ | BbhNavigate37BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT690 |
| ✓ | BbhNavigate38BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 237 |
| ✓ | BbhNavigate39BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT570 |
| ✓ | BbhNavigate40BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT704 |
| ✓ | BbhNavigate41BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 248 |
| ✓ | BbhNavigate42BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 863 |
| ✓ | BbhNavigate43BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT2 113 |
| ✓ | BbhNavigate44BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT622 |
| ✓ | BbhNavigate45BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT956 |
| ✓ | BbhNavigate46BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT4 710 |
| ✓ | BbhNavigate47BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 301 |
| ✓ | BbhNavigate48BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT916 |
| ✓ | BbhNavigate49BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 151 |
| ✓ | BbhNavigate50BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 008 |
| ✓ | BbhNavigate51BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 296 |
| ✓ | BbhNavigate52BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT1 404 |
| ✓ | BbhNavigate53BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 036 |
| ✓ | BbhNavigate54BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT2 339 |
Reasoning - BBH navigation - returns to start? (set 2)
30 passed |
100.0% |
IN30 127 OUT42 848 | |
| ✓ | BbhNavigate55BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 551 |
| ✓ | BbhNavigate56BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 794 |
| ✓ | BbhNavigate57BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT716 |
| ✓ | BbhNavigate58BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 038 |
| ✓ | BbhNavigate59BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 935 |
| ✓ | BbhNavigate60BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT871 |
| ✓ | BbhNavigate61BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT3 043 |
| ✓ | BbhNavigate62BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 289 |
| ✓ | BbhNavigate63BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 914 |
| ✓ | BbhNavigate64BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT661 |
| ✓ | BbhNavigate65BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT694 |
| ✓ | BbhNavigate66BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 567 |
| ✓ | BbhNavigate67BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 317 |
| ✓ | BbhNavigate68BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 508 |
| ✓ | BbhNavigate69BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT698 |
| ✓ | BbhNavigate70BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT496 |
| ✓ | BbhNavigate71BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT579 |
| ✓ | BbhNavigate72BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT880 |
| ✓ | BbhNavigate73BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT642 |
| ✓ | BbhNavigate74BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT2 756 |
| ✓ | BbhNavigate75BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT554 |
| ✓ | BbhNavigate76BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 013 |
| ✓ | BbhNavigate77BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT2 918 |
| ✓ | BbhNavigate78BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT2 869 |
| ✓ | BbhNavigate79BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT599 |
| ✓ | BbhNavigate80BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT2 060 |
| ✓ | BbhNavigate81BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 535 |
| ✓ | BbhNavigate82BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT641 |
| ✓ | BbhNavigate83BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 707 |
| ✓ | BbhNavigate84BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 003 |
Reasoning - BBH navigation - returns to start? (set 3)
32 passed |
100.0% |
IN32 149 OUT47 971 | |
| ✓ | BbhNavigate100CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 935 |
| ✓ | BbhNavigate101CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 671 |
| ✓ | BbhNavigate102CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 530 |
| ✓ | BbhNavigate103CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT974 |
| ✓ | BbhNavigate104CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 186 |
| ✓ | BbhNavigate105CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT3 581 |
| ✓ | BbhNavigate106CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 443 |
| ✓ | BbhNavigate107CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT768 |
| ✓ | BbhNavigate108CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 171 |
| ✓ | BbhNavigate109CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT2 601 |
| ✓ | BbhNavigate110CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 829 |
| ✓ | BbhNavigate111CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT1 343 |
| ✓ | BbhNavigate112CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 725 |
| ✓ | BbhNavigate113CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT876 |
| ✓ | BbhNavigate114CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 251 |
| ✓ | BbhNavigate115CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 137 |
| ✓ | BbhNavigate116CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT598 |
| ✓ | BbhNavigate85CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 007 |
| ✓ | BbhNavigate86CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT3 400 |
| ✓ | BbhNavigate87CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT710 |
| ✓ | BbhNavigate88CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 065 |
| ✓ | BbhNavigate89CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT958 |
| ✓ | BbhNavigate90CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT3 743 |
| ✓ | BbhNavigate91CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT2 093 |
| ✓ | BbhNavigate92CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 871 |
| ✓ | BbhNavigate93CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 395 |
| ✓ | BbhNavigate94CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 499 |
| ✓ | BbhNavigate95CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT671 |
| ✓ | BbhNavigate96CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT925 |
| ✓ | BbhNavigate97CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 078 |
| ✓ | BbhNavigate98CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT1 185 |
| ✓ | BbhNavigate99CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT752 |
Reasoning - BBH navigation - returns to start? (set 4)
32 passed |
100.0% |
IN32 137 OUT44 836 | |
| ✓ | BbhNavigate117CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 329 |
| ✓ | BbhNavigate118CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 042 |
| ✓ | BbhNavigate119CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT645 |
| ✓ | BbhNavigate120CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 373 |
| ✓ | BbhNavigate121CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT958 |
| ✓ | BbhNavigate122CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT891 |
| ✓ | BbhNavigate123CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT761 |
| ✓ | BbhNavigate124CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT2 480 |
| ✓ | BbhNavigate125CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 177 |
| ✓ | BbhNavigate126CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT3 040 |
| ✓ | BbhNavigate127CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 819 |
| ✓ | BbhNavigate128CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 387 |
| ✓ | BbhNavigate129CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 232 |
| ✓ | BbhNavigate130CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 526 |
| ✓ | BbhNavigate131CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 732 |
| ✓ | BbhNavigate132CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 031 |
| ✓ | BbhNavigate133CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 314 |
| ✓ | BbhNavigate134CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT523 |
| ✓ | BbhNavigate135CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 550 |
| ✓ | BbhNavigate136CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT2 196 |
| ✓ | BbhNavigate137CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT2 137 |
| ✓ | BbhNavigate138CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 429 |
| ✓ | BbhNavigate139CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT2 325 |
| ✓ | BbhNavigate140CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 085 |
| ✓ | BbhNavigate141CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 639 |
| ✓ | BbhNavigate142CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 495 |
| ✓ | BbhNavigate143CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 288 |
| ✓ | BbhNavigate144CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT752 |
| ✓ | BbhNavigate145CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 483 |
| ✓ | BbhNavigate146CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 102 |
| ✓ | BbhNavigate147CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 106 |
| ✓ | BbhNavigate148CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT989 |
Reasoning - BBH navigation - returns to start? (set 5)
32 passed |
100.0% |
IN32 074 OUT72 024 | |
| ✓ | BbhNavigate149CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT2 345 |
| ✓ | BbhNavigate150CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 415 |
| ✓ | BbhNavigate151CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT629 |
| ✓ | BbhNavigate152CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT561 |
| ✓ | BbhNavigate153CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 048 |
| ✓ | BbhNavigate154CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT3 495 |
| ✓ | BbhNavigate155CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT829 |
| ✓ | BbhNavigate156CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 367 |
| ✓ | BbhNavigate157CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 019 |
| ✓ | BbhNavigate158CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT3 112 |
| ✓ | BbhNavigate159CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 048 |
| ✓ | BbhNavigate160CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT710 |
| ✓ | BbhNavigate161CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT788 |
| ✓ | BbhNavigate162CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 455 |
| ✓ | BbhNavigate163CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT1 711 |
| ✓ | BbhNavigate164CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT2 105 |
| ✓ | BbhNavigate165CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT1 083 |
| ✓ | BbhNavigate166CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 432 |
| ✓ | BbhNavigate167CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 109 |
| ✓ | BbhNavigate168CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 219 |
| ✓ | BbhNavigate169CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT893 |
| ✓ | BbhNavigate170CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT745 |
| ✓ | BbhNavigate171CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 728 |
| ✓ | BbhNavigate172CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT8 325 |
| ✓ | BbhNavigate173CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT2 933 |
| ✓ | BbhNavigate174CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 417 |
| ✓ | BbhNavigate175CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT5 095 |
| ✓ | BbhNavigate176CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 408 |
| ✓ | BbhNavigate177CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT9 231 |
| ✓ | BbhNavigate178CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT4 368 |
| ✓ | BbhNavigate179CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 663 |
| ✓ | BbhNavigate180CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT2 738 |
Reasoning - BBH navigation - returns to start? (set 6)
32 passed |
100.0% |
IN32 152 OUT56 681 | |
| ✓ | BbhNavigate181CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 055 |
| ✓ | BbhNavigate182CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 979 |
| ✓ | BbhNavigate183CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT831 |
| ✓ | BbhNavigate184CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT1 785 |
| ✓ | BbhNavigate185CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT949 |
| ✓ | BbhNavigate186CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT2 303 |
| ✓ | BbhNavigate187CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT2 984 |
| ✓ | BbhNavigate188CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 373 |
| ✓ | BbhNavigate189CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT5 820 |
| ✓ | BbhNavigate190CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT470 |
| ✓ | BbhNavigate191CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT998 |
| ✓ | BbhNavigate192CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT3 041 |
| ✓ | BbhNavigate193CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 790 |
| ✓ | BbhNavigate194CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT776 |
| ✓ | BbhNavigate195CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 026 |
| ✓ | BbhNavigate196CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT3 012 |
| ✓ | BbhNavigate197CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT3 930 |
| ✓ | BbhNavigate198CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 120 |
| ✓ | BbhNavigate199CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT709 |
| ✓ | BbhNavigate200CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT2 778 |
| ✓ | BbhNavigate201CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 566 |
| ✓ | BbhNavigate202CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 457 |
| ✓ | BbhNavigate203CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 143 |
| ✓ | BbhNavigate204CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 047 |
| ✓ | BbhNavigate205CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT889 |
| ✓ | BbhNavigate206CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 763 |
| ✓ | BbhNavigate207CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT2 746 |
| ✓ | BbhNavigate208CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 252 |
| ✓ | BbhNavigate209CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 117 |
| ✓ | BbhNavigate210CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT986 |
| ✓ | BbhNavigate211CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 784 |
| ✓ | BbhNavigate212CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT1 202 |
Reasoning - BBH navigation - returns to start? (set 7)
30 passed |
100.0% |
IN30 053 OUT39 493 | |
| ✓ | BbhNavigate213CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 868 |
| ✓ | BbhNavigate214CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 595 |
| ✓ | BbhNavigate215CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT2 080 |
| ✓ | BbhNavigate216CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 027 |
| ✓ | BbhNavigate217CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT892 |
| ✓ | BbhNavigate218CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 673 |
| ✓ | BbhNavigate219CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 641 |
| ✓ | BbhNavigate220CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 429 |
| ✓ | BbhNavigate221CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 156 |
| ✓ | BbhNavigate222CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT576 |
| ✓ | BbhNavigate223CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT1 465 |
| ✓ | BbhNavigate224CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 210 |
| ✓ | BbhNavigate225CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT758 |
| ✓ | BbhNavigate226CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT474 |
| ✓ | BbhNavigate227CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT856 |
| ✓ | BbhNavigate228CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT568 |
| ✓ | BbhNavigate229CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 189 |
| ✓ | BbhNavigate230CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT691 |
| ✓ | BbhNavigate231CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT3 596 |
| ✓ | BbhNavigate232CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT829 |
| ✓ | BbhNavigate233CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT352 |
| ✓ | BbhNavigate234CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT2 121 |
| ✓ | BbhNavigate235CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT3 226 |
| ✓ | BbhNavigate236CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT430 |
| ✓ | BbhNavigate237CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT904 |
| ✓ | BbhNavigate238CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 070 |
| ✓ | BbhNavigate239CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 337 |
| ✓ | BbhNavigate240CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT662 |
| ✓ | BbhNavigate241CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT1 174 |
| ✓ | BbhNavigate242CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 644 |
Reasoning - BBH object counting (set 1)
32 passed |
100.0% |
IN31 706 OUT22 924 | |
| ✓ | BbhNavigate243CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 034 OUT3 028 |
| ✓ | BbhNavigate244CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT1 799 |
| ✓ | BbhNavigate245CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 414 |
| ✓ | BbhNavigate246CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT682 |
| ✓ | BbhNavigate247CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 074 |
| ✓ | BbhNavigate248CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT1 349 |
| ✓ | BbhNavigate249CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 043 |
| ✓ | BbhObjectCounting10CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT428 |
| ✓ | BbhObjectCounting11CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT838 |
| ✓ | BbhObjectCounting12CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT132 |
| ✓ | BbhObjectCounting13CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT443 |
| ✓ | BbhObjectCounting14CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT307 |
| ✓ | BbhObjectCounting15CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT528 |
| ✓ | BbhObjectCounting16CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT614 |
| ✓ | BbhObjectCounting17CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT415 |
| ✓ | BbhObjectCounting18CBI have a blackberry, a plum, and a banana. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT208 |
| ✓ | BbhObjectCounting19CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT425 |
| ✓ | BbhObjectCounting20CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT249 |
| ✓ | BbhObjectCounting21CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT529 |
| ✓ | BbhObjectCounting22CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT372 |
| ✓ | BbhObjectCounting23CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT964 |
| ✓ | BbhObjectCounting24CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT511 |
| ✓ | BbhObjectCounting25CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT736 |
| ✓ | BbhObjectCounting26CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT132 |
| ✓ | BbhObjectCounting27CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT414 |
| ✓ | BbhObjectCounting28CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT724 |
| ✓ | BbhObjectCounting29CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT660 |
| ✓ | BbhObjectCounting30CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT566 |
| ✓ | BbhObjectCounting31CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT681 |
| ✓ | BbhObjectCounting32CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT657 |
| ✓ | BbhObjectCounting8CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT615 |
| ✓ | BbhObjectCounting9CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT387 |
Reasoning - BBH object counting (set 2)
32 passed |
100.0% |
IN31 517 OUT14 734 | |
| ✓ | BbhObjectCounting33CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT204 |
| ✓ | BbhObjectCounting34CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN970 OUT102 |
| ✓ | BbhObjectCounting35CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT568 |
| ✓ | BbhObjectCounting36CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT242 |
| ✓ | BbhObjectCounting37CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT208 |
| ✓ | BbhObjectCounting38CBI have a raspberry, a grape, and an orange. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT95 |
| ✓ | BbhObjectCounting39CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT630 |
| ✓ | BbhObjectCounting40CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT674 |
| ✓ | BbhObjectCounting41CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT465 |
| ✓ | BbhObjectCounting42CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT520 |
| ✓ | BbhObjectCounting43CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT599 |
| ✓ | BbhObjectCounting44CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT210 |
| ✓ | BbhObjectCounting45CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT192 |
| ✓ | BbhObjectCounting46CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT488 |
| ✓ | BbhObjectCounting47CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT561 |
| ✓ | BbhObjectCounting48CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT396 |
| ✓ | BbhObjectCounting49CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT934 |
| ✓ | BbhObjectCounting50CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT718 |
| ✓ | BbhObjectCounting51CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT473 |
| ✓ | BbhObjectCounting52CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT861 |
| ✓ | BbhObjectCounting53CBI have a frog, and a mouse. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT198 |
| ✓ | BbhObjectCounting54CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT664 |
| ✓ | BbhObjectCounting55CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT249 |
| ✓ | BbhObjectCounting56CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT526 |
| ✓ | BbhObjectCounting57CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT914 |
| ✓ | BbhObjectCounting58CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT680 |
| ✓ | BbhObjectCounting59CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT382 |
| ✓ | BbhObjectCounting60CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT732 |
| ✓ | BbhObjectCounting61CBI have a rabbit, a frog, and a snail. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT107 |
| ✓ | BbhObjectCounting62CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT516 |
| ✓ | BbhObjectCounting63CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT262 |
| ✓ | BbhObjectCounting64CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT364 |
Reasoning - BBH object counting (set 3)
32 passed |
100.0% |
IN31 587 OUT18 122 | |
| ✓ | BbhObjectCounting65CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT451 |
| ✓ | BbhObjectCounting66CBI have a grape, and a plum. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT180 |
| ✓ | BbhObjectCounting67CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT576 |
| ✓ | BbhObjectCounting68CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT115 |
| ✓ | BbhObjectCounting69CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT405 |
| ✓ | BbhObjectCounting70CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT779 |
| ✓ | BbhObjectCounting71CBI have a dog, a pig, and a snail. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT444 |
| ✓ | BbhObjectCounting72CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT529 |
| ✓ | BbhObjectCounting73CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT770 |
| ✓ | BbhObjectCounting74CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT802 |
| ✓ | BbhObjectCounting75CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT716 |
| ✓ | BbhObjectCounting76CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT587 |
| ✓ | BbhObjectCounting77CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT379 |
| ✓ | BbhObjectCounting78CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT424 |
| ✓ | BbhObjectCounting79CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT658 |
| ✓ | BbhObjectCounting80CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT358 |
| ✓ | BbhObjectCounting81CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT279 |
| ✓ | BbhObjectCounting82CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT949 |
| ✓ | BbhObjectCounting83CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT725 |
| ✓ | BbhObjectCounting84CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT262 |
| ✓ | BbhObjectCounting85CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT752 |
| ✓ | BbhObjectCounting86CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 682 |
| ✓ | BbhObjectCounting87CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT531 |
| ✓ | BbhObjectCounting88CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT520 |
| ✓ | BbhObjectCounting89CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT447 |
| ✓ | BbhObjectCounting90CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT431 |
| ✓ | BbhObjectCounting91CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT590 |
| ✓ | BbhObjectCounting92CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 060 |
| ✓ | BbhObjectCounting93CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT209 |
| ✓ | BbhObjectCounting94CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT712 |
| ✓ | BbhObjectCounting95CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT330 |
| ✓ | BbhObjectCounting96CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT470 |
Reasoning - BBH object counting (set 4)
31 passed |
100.0% |
IN30 460 OUT17 638 | |
| ✓ | BbhObjectCounting100CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT732 |
| ✓ | BbhObjectCounting101CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT435 |
| ✓ | BbhObjectCounting102CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT637 |
| ✓ | BbhObjectCounting103CBI have a table, and three cars. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT253 |
| ✓ | BbhObjectCounting104CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT578 |
| ✓ | BbhObjectCounting105CBI have a microwave, a bed, and a couch. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT286 |
| ✓ | BbhObjectCounting106CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT177 |
| ✓ | BbhObjectCounting107CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 996 |
| ✓ | BbhObjectCounting108CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT506 |
| ✓ | BbhObjectCounting109CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT1 104 |
| ✓ | BbhObjectCounting110CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT620 |
| ✓ | BbhObjectCounting111CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT277 |
| ✓ | BbhObjectCounting112CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT697 |
| ✓ | BbhObjectCounting113CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT441 |
| ✓ | BbhObjectCounting114CBI have a donkey, a goat, and a cow. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT173 |
| ✓ | BbhObjectCounting115CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT128 |
| ✓ | BbhObjectCounting116CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT533 |
| ✓ | BbhObjectCounting117CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT621 |
| ✓ | BbhObjectCounting118CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT305 |
| ✓ | BbhObjectCounting119CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT883 |
| ✓ | BbhObjectCounting120CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT879 |
| ✓ | BbhObjectCounting121CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT395 |
| ✓ | BbhObjectCounting122CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT523 |
| ✓ | BbhObjectCounting123CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT543 |
| ✓ | BbhObjectCounting124CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT519 |
| ✓ | BbhObjectCounting125CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT546 |
| ✓ | BbhObjectCounting126CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT251 |
| ✓ | BbhObjectCounting127CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT669 |
| ✓ | BbhObjectCounting97CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 094 |
| ✓ | BbhObjectCounting98CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT245 |
| ✓ | BbhObjectCounting99CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT592 |
Reasoning - BBH object counting (set 5)
31 passed |
100.0% |
IN30 405 OUT13 169 | |
| ✓ | BbhObjectCounting128CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT202 |
| ✓ | BbhObjectCounting129CBI have two grapes, and a plum. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT285 |
| ✓ | BbhObjectCounting130CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT114 |
| ✓ | BbhObjectCounting131CBI have a pig, and a donkey. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN970 OUT189 |
| ✓ | BbhObjectCounting132CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT443 |
| ✓ | BbhObjectCounting133CBI have a microwave, and a bed. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT189 |
| ✓ | BbhObjectCounting134CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT458 |
| ✓ | BbhObjectCounting135CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT607 |
| ✓ | BbhObjectCounting136CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT226 |
| ✓ | BbhObjectCounting137CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT275 |
| ✓ | BbhObjectCounting138CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT301 |
| ✓ | BbhObjectCounting139CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT684 |
| ✓ | BbhObjectCounting140CBI have a chair, and two microwaves. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT288 |
| ✓ | BbhObjectCounting141CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT559 |
| ✓ | BbhObjectCounting142CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT279 |
| ✓ | BbhObjectCounting143CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT695 |
| ✓ | BbhObjectCounting144CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT652 |
| ✓ | BbhObjectCounting145CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT279 |
| ✓ | BbhObjectCounting146CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT414 |
| ✓ | BbhObjectCounting147CBI have a cat, and a fish. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT423 |
| ✓ | BbhObjectCounting148CBI have a head of broccoli, and a yam. How many vegetables do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT268 |
| ✓ | BbhObjectCounting149CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT388 |
| ✓ | BbhObjectCounting150CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT524 |
| ✓ | BbhObjectCounting151CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT262 |
| ✓ | BbhObjectCounting152CBI have four snails, and a cow. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN970 OUT426 |
| ✓ | BbhObjectCounting153CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT532 |
| ✓ | BbhObjectCounting154CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT133 |
| ✓ | BbhObjectCounting155CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT542 |
| ✓ | BbhObjectCounting156CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 660 |
| ✓ | BbhObjectCounting157CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT298 |
| ✓ | BbhObjectCounting158CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT574 |
Reasoning - BBH object counting (set 6)
31 passed1 failed |
96.9% |
IN31 480 OUT16 119 | |
| × | BbhObjectCounting159CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "8")
Context: BbhObjectCounting159CC: expected 8, got [6] |
Failed | IN987 OUT1 124 |
| ✓ | BbhObjectCounting160CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT138 |
| ✓ | BbhObjectCounting161CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT465 |
| ✓ | BbhObjectCounting162CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT301 |
| ✓ | BbhObjectCounting163CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT761 |
| ✓ | BbhObjectCounting164CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT550 |
| ✓ | BbhObjectCounting165CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT967 |
| ✓ | BbhObjectCounting166CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT341 |
| ✓ | BbhObjectCounting167CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT612 |
| ✓ | BbhObjectCounting168CCI have a toaster, a table, and a fridge. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT201 |
| ✓ | BbhObjectCounting169CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT523 |
| ✓ | BbhObjectCounting170CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT644 |
| ✓ | BbhObjectCounting171CCI have a garlic, and an onion. How many vegetables do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT320 |
| ✓ | BbhObjectCounting172CCI have a grape, a banana, and a blackberry. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT182 |
| ✓ | BbhObjectCounting173CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT699 |
| ✓ | BbhObjectCounting174CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT727 |
| ✓ | BbhObjectCounting175CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT924 |
| ✓ | BbhObjectCounting176CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT245 |
| ✓ | BbhObjectCounting177CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT494 |
| ✓ | BbhObjectCounting178CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT198 |
| ✓ | BbhObjectCounting179CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT917 |
| ✓ | BbhObjectCounting180CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT238 |
| ✓ | BbhObjectCounting181CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT399 |
| ✓ | BbhObjectCounting182CCI have a microwave, and a chair. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT177 |
| ✓ | BbhObjectCounting183CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT574 |
| ✓ | BbhObjectCounting184CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT617 |
| ✓ | BbhObjectCounting185CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT408 |
| ✓ | BbhObjectCounting186CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT818 |
| ✓ | BbhObjectCounting187CCI have a grape, an apple, and a banana. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT248 |
| ✓ | BbhObjectCounting188CCI have two blackberries, and a grape. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN970 OUT216 |
| ✓ | BbhObjectCounting189CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT497 |
| ✓ | BbhObjectCounting190CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT594 |
Reasoning - BBH object counting (set 7)
31 passed1 failed |
96.9% |
IN31 557 OUT16 013 | |
| ✓ | BbhObjectCounting191CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT680 |
| ✓ | BbhObjectCounting192CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT214 |
| ✓ | BbhObjectCounting193CCI have a dog, a chicken, and four snails. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT468 |
| ✓ | BbhObjectCounting194CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 042 |
| ✓ | BbhObjectCounting195CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT763 |
| ✓ | BbhObjectCounting196CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT140 |
| ✓ | BbhObjectCounting197CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT612 |
| ✓ | BbhObjectCounting198CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT700 |
| ✓ | BbhObjectCounting199CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT761 |
| ✓ | BbhObjectCounting200CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT161 |
| ✓ | BbhObjectCounting201CCI have an apple, and two blackberries. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN970 OUT253 |
| ✓ | BbhObjectCounting202CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT422 |
| ✓ | BbhObjectCounting203CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT708 |
| ✓ | BbhObjectCounting204CCI have two lamps, and two microwaves. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT106 |
| ✓ | BbhObjectCounting205CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT515 |
| ✓ | BbhObjectCounting206CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT901 |
| ✓ | BbhObjectCounting207CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT205 |
| × | BbhObjectCounting208CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "10")
Context: BbhObjectCounting208CC: expected 10, got [9] |
Failed | IN987 OUT327 |
| ✓ | BbhObjectCounting209CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT917 |
| ✓ | BbhObjectCounting210CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT182 |
| ✓ | BbhObjectCounting211CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT561 |
| ✓ | BbhObjectCounting212CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT403 |
| ✓ | BbhObjectCounting213CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT165 |
| ✓ | BbhObjectCounting214CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT632 |
| ✓ | BbhObjectCounting215CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT388 |
| ✓ | BbhObjectCounting216CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT491 |
| ✓ | BbhObjectCounting217CCI have a toaster, a car, and a table. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT181 |
| ✓ | BbhObjectCounting218CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 661 |
| ✓ | BbhObjectCounting219CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT575 |
| ✓ | BbhObjectCounting220CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT261 |
| ✓ | BbhObjectCounting221CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT226 |
| ✓ | BbhObjectCounting222CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT392 |
Reasoning - BBH object counting (set 8)
26 passed1 failed |
96.3% |
IN26 635 OUT14 781 | |
| ✓ | BbhObjectCounting223CCI have an oven, a bed, and a toaster. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT107 |
| ✓ | BbhObjectCounting224CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 537 |
| ✓ | BbhObjectCounting225CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT259 |
| ✓ | BbhObjectCounting226CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT400 |
| ✓ | BbhObjectCounting227CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT288 |
| ✓ | BbhObjectCounting228CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT842 |
| ✓ | BbhObjectCounting229CCI have two cars, and an oven. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT382 |
| ✓ | BbhObjectCounting230CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT918 |
| ✓ | BbhObjectCounting231CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT740 |
| × | BbhObjectCounting232CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "14")
Context: BbhObjectCounting232CC: expected 14, got [13] |
Failed | IN997 OUT584 |
| ✓ | BbhObjectCounting233CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT533 |
| ✓ | BbhObjectCounting234CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT115 |
| ✓ | BbhObjectCounting235CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT721 |
| ✓ | BbhObjectCounting236CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT322 |
| ✓ | BbhObjectCounting237CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT588 |
| ✓ | BbhObjectCounting238CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT540 |
| ✓ | BbhObjectCounting239CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT965 |
| ✓ | BbhObjectCounting240CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT973 |
| ✓ | BbhObjectCounting241CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT248 |
| ✓ | BbhObjectCounting242CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT337 |
| ✓ | BbhObjectCounting243CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT520 |
| ✓ | BbhObjectCounting244CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT590 |
| ✓ | BbhObjectCounting245CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT535 |
| ✓ | BbhObjectCounting246CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT204 |
| ✓ | BbhObjectCounting247CCI have a fish, a snail, and two rabbits. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT304 |
| ✓ | BbhObjectCounting248CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT463 |
| ✓ | BbhObjectCounting249CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT766 |
Reasoning - BBH penguins in a table (set 1)
32 passed1 failed |
97.0% |
IN37 978 OUT25 665 | |
| ✓ | BbhPenguinsInATable10CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 018 |
| ✓ | BbhPenguinsInATable11CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 201 OUT554 |
| ✓ | BbhPenguinsInATable12CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT843 |
| ✓ | BbhPenguinsInATable13CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 224 OUT555 |
| ✓ | BbhPenguinsInATable14CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT593 |
| ✓ | BbhPenguinsInATable15CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 934 |
| ✓ | BbhPenguinsInATable16CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 226 OUT506 |
| ✓ | BbhPenguinsInATable17CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT456 |
| ✓ | BbhPenguinsInATable18CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT581 |
| ✓ | BbhPenguinsInATable19CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT1 958 |
| ✓ | BbhPenguinsInATable20CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT583 |
| ✓ | BbhPenguinsInATable21CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT1 291 |
| ✓ | BbhPenguinsInATable22CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT568 |
| ✓ | BbhPenguinsInATable23CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT1 208 |
| ✓ | BbhPenguinsInATable24CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT362 |
| ✓ | BbhPenguinsInATable25CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 121 OUT705 |
| ✓ | BbhPenguinsInATable26CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 218 OUT623 |
| ✓ | BbhPenguinsInATable27CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT829 |
| ✓ | BbhPenguinsInATable28CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT406 |
| ✓ | BbhPenguinsInATable29CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 220 OUT430 |
| ✓ | BbhPenguinsInATable30CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 218 OUT737 |
| ✓ | BbhPenguinsInATable31CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 225 OUT1 234 |
| ✓ | BbhPenguinsInATable32CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 193 OUT478 |
| ✓ | BbhPenguinsInATable33CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 195 OUT496 |
| × | BbhPenguinsInATable34CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhPenguinsInATable34CC: expected (A), got [d]
Values:
got = "d" |
Failed | IN1 104 OUT1 686 |
| ✓ | BbhPenguinsInATable35CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT324 |
| ✓ | BbhPenguinsInATable36CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT636 |
| ✓ | BbhPenguinsInATable37CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT434 |
| ✓ | BbhPenguinsInATable38CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 199 OUT280 |
| ✓ | BbhPenguinsInATable39CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 044 |
| ✓ | BbhPenguinsInATable40CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT1 020 |
| ✓ | BbhPenguinsInATable8CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT522 |
| ✓ | BbhPenguinsInATable9CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT771 |
Reasoning - BBH penguins in a table (set 2)
34 passed |
100.0% |
IN38 629 OUT22 380 | |
| ✓ | BbhPenguinsInATable41CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT550 |
| ✓ | BbhPenguinsInATable42CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 033 |
| ✓ | BbhPenguinsInATable43CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 224 OUT1 295 |
| ✓ | BbhPenguinsInATable44CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT682 |
| ✓ | BbhPenguinsInATable45CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT360 |
| ✓ | BbhPenguinsInATable46CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 217 OUT411 |
| ✓ | BbhPenguinsInATable47CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 198 OUT304 |
| ✓ | BbhPenguinsInATable48CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT835 |
| ✓ | BbhPenguinsInATable49CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT298 |
| ✓ | BbhPenguinsInATable50CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 199 OUT586 |
| ✓ | BbhPenguinsInATable51CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT870 |
| ✓ | BbhPenguinsInATable52CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT420 |
| ✓ | BbhPenguinsInATable53CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT497 |
| ✓ | BbhPenguinsInATable54CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 138 OUT1 006 |
| ✓ | BbhPenguinsInATable55CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT372 |
| ✓ | BbhPenguinsInATable56CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT686 |
| ✓ | BbhPenguinsInATable57CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT654 |
| ✓ | BbhPenguinsInATable58CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT569 |
| ✓ | BbhPenguinsInATable59CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT534 |
| ✓ | BbhPenguinsInATable60CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT506 |
| ✓ | BbhPenguinsInATable61CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT486 |
| ✓ | BbhPenguinsInATable62CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT860 |
| ✓ | BbhPenguinsInATable63CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT451 |
| ✓ | BbhPenguinsInATable64CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 228 OUT1 188 |
| ✓ | BbhPenguinsInATable65CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 123 OUT902 |
| ✓ | BbhPenguinsInATable66CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT815 |
| ✓ | BbhPenguinsInATable67CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 223 OUT698 |
| ✓ | BbhPenguinsInATable68CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 286 |
| ✓ | BbhPenguinsInATable69CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT304 |
| ✓ | BbhPenguinsInATable70CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT698 |
| ✓ | BbhPenguinsInATable71CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT577 |
| ✓ | BbhPenguinsInATable72CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT404 |
| ✓ | BbhPenguinsInATable73CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT472 |
| ✓ | BbhPenguinsInATable74CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 222 OUT771 |
Reasoning - BBH penguins in a table (set 3)
36 passed |
100.0% |
IN41 003 OUT24 939 | |
| ✓ | BbhPenguins100CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT866 |
| ✓ | BbhPenguins101CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT531 |
| ✓ | BbhPenguins102CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT687 |
| ✓ | BbhPenguins103CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT436 |
| ✓ | BbhPenguins104CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 197 OUT628 |
| ✓ | BbhPenguins105CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT815 |
| ✓ | BbhPenguins106CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT483 |
| ✓ | BbhPenguins107CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT737 |
| ✓ | BbhPenguins108CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT773 |
| ✓ | BbhPenguins109CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT697 |
| ✓ | BbhPenguins110CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT615 |
| ✓ | BbhPenguins75CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 222 OUT626 |
| ✓ | BbhPenguins76CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT756 |
| ✓ | BbhPenguins77CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 197 OUT339 |
| ✓ | BbhPenguins78CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT1 154 |
| ✓ | BbhPenguins79CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT1 008 |
| ✓ | BbhPenguins80CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 193 OUT449 |
| ✓ | BbhPenguins81CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT659 |
| ✓ | BbhPenguins82CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT1 273 |
| ✓ | BbhPenguins83CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT878 |
| ✓ | BbhPenguins84CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 208 OUT825 |
| ✓ | BbhPenguins85CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 078 |
| ✓ | BbhPenguins86CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT359 |
| ✓ | BbhPenguins87CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT571 |
| ✓ | BbhPenguins88CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT787 |
| ✓ | BbhPenguins89CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 223 OUT922 |
| ✓ | BbhPenguins90CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT502 |
| ✓ | BbhPenguins91CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT617 |
| ✓ | BbhPenguins92CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT713 |
| ✓ | BbhPenguins93CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT507 |
| ✓ | BbhPenguins94CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT647 |
| ✓ | BbhPenguins95CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT365 |
| ✓ | BbhPenguins96CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 229 OUT988 |
| ✓ | BbhPenguins97CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT416 |
| ✓ | BbhPenguins98CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 190 OUT833 |
| ✓ | BbhPenguins99CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT399 |
Reasoning - BBH penguins in a table (set 4)
35 passed |
100.0% |
IN39 908 OUT22 382 | |
| ✓ | BbhPenguins111CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT350 |
| ✓ | BbhPenguins112CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT448 |
| ✓ | BbhPenguins113CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 198 OUT1 097 |
| ✓ | BbhPenguins114CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 200 OUT732 |
| ✓ | BbhPenguins115CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT386 |
| ✓ | BbhPenguins116CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT478 |
| ✓ | BbhPenguins117CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 225 OUT664 |
| ✓ | BbhPenguins118CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT485 |
| ✓ | BbhPenguins119CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT302 |
| ✓ | BbhPenguins120CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT916 |
| ✓ | BbhPenguins121CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT525 |
| ✓ | BbhPenguins122CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 204 OUT545 |
| ✓ | BbhPenguins123CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT551 |
| ✓ | BbhPenguins124CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT264 |
| ✓ | BbhPenguins125CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 192 OUT486 |
| ✓ | BbhPenguins126CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 177 |
| ✓ | BbhPenguins127CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT1 822 |
| ✓ | BbhPenguins128CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 198 OUT740 |
| ✓ | BbhPenguins129CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT325 |
| ✓ | BbhPenguins130CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT940 |
| ✓ | BbhPenguins131CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT526 |
| ✓ | BbhPenguins132CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 230 OUT368 |
| ✓ | BbhPenguins133CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 203 OUT778 |
| ✓ | BbhPenguins134CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT748 |
| ✓ | BbhPenguins135CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT403 |
| ✓ | BbhPenguins136CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT651 |
| ✓ | BbhPenguins137CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT441 |
| ✓ | BbhPenguins138CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT355 |
| ✓ | BbhPenguins139CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT1 214 |
| ✓ | BbhPenguins140CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT764 |
| ✓ | BbhPenguins141CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 223 OUT442 |
| ✓ | BbhPenguins142CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT391 |
| ✓ | BbhPenguins143CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 134 |
| ✓ | BbhPenguins144CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT534 |
| ✓ | BbhPenguins145CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT400 |
Reasoning - BBH reasoning about colored objects (set 1)
33 passed |
100.0% |
IN35 191 OUT28 309 | |
| ✓ | BbhColoredObjects10CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT872 |
| ✓ | BbhColoredObjects11CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT618 |
| ✓ | BbhColoredObjects12CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT692 |
| ✓ | BbhColoredObjects13CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT869 |
| ✓ | BbhColoredObjects14CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT1 672 |
| ✓ | BbhColoredObjects15CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT345 |
| ✓ | BbhColoredObjects16CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT1 524 |
| ✓ | BbhColoredObjects17CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT510 |
| ✓ | BbhColoredObjects18CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT837 |
| ✓ | BbhColoredObjects19CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT963 |
| ✓ | BbhColoredObjects20CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT667 |
| ✓ | BbhColoredObjects21CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT1 267 |
| ✓ | BbhColoredObjects22CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT507 |
| ✓ | BbhColoredObjects23CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT430 |
| ✓ | BbhColoredObjects24CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT2 658 |
| ✓ | BbhColoredObjects25CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT690 |
| ✓ | BbhColoredObjects26CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT742 |
| ✓ | BbhColoredObjects27CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT920 |
| ✓ | BbhColoredObjects28CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT915 |
| ✓ | BbhColoredObjects29CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT461 |
| ✓ | BbhColoredObjects30CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT471 |
| ✓ | BbhColoredObjects31CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT616 |
| ✓ | BbhColoredObjects32CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT652 |
| ✓ | BbhColoredObjects33CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT855 |
| ✓ | BbhColoredObjects34CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT401 |
| ✓ | BbhColoredObjects35CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT966 |
| ✓ | BbhColoredObjects36CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT1 042 |
| ✓ | BbhColoredObjects37CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT598 |
| ✓ | BbhColoredObjects38CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT577 |
| ✓ | BbhColoredObjects39CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT586 |
| ✓ | BbhColoredObjects40CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT1 416 |
| ✓ | BbhColoredObjects8CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT424 |
| ✓ | BbhColoredObjects9CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT1 546 |
Reasoning - BBH reasoning about colored objects (set 2)
30 passed |
100.0% |
IN31 782 OUT22 161 | |
| ✓ | BbhColoredObjects41CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT449 |
| ✓ | BbhColoredObjects42CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT723 |
| ✓ | BbhColoredObjects43CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT190 |
| ✓ | BbhColoredObjects44CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT743 |
| ✓ | BbhColoredObjects45CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT142 |
| ✓ | BbhColoredObjects46CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT235 |
| ✓ | BbhColoredObjects47CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT520 |
| ✓ | BbhColoredObjects48CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT930 |
| ✓ | BbhColoredObjects49CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 225 |
| ✓ | BbhColoredObjects50CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT638 |
| ✓ | BbhColoredObjects51CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT494 |
| ✓ | BbhColoredObjects52CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT671 |
| ✓ | BbhColoredObjects53CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT511 |
| ✓ | BbhColoredObjects54CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT649 |
| ✓ | BbhColoredObjects55CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT716 |
| ✓ | BbhColoredObjects56CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 108 |
| ✓ | BbhColoredObjects57CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 007 |
| ✓ | BbhColoredObjects58CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT804 |
| ✓ | BbhColoredObjects59CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT793 |
| ✓ | BbhColoredObjects60CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT1 233 |
| ✓ | BbhColoredObjects61CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 041 OUT1 012 |
| ✓ | BbhColoredObjects62CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT440 |
| ✓ | BbhColoredObjects63CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT991 |
| ✓ | BbhColoredObjects64CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT912 |
| ✓ | BbhColoredObjects65CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT855 |
| ✓ | BbhColoredObjects66CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT1 681 |
| ✓ | BbhColoredObjects67CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT601 |
| ✓ | BbhColoredObjects68CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT873 |
| ✓ | BbhColoredObjects69CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT817 |
| ✓ | BbhColoredObjects70CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT198 |
Reasoning - BBH reasoning about colored objects (set 3)
24 passed |
100.0% |
IN25 610 OUT20 637 | |
| ✓ | BbhColoredObjects71CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT256 |
| ✓ | BbhColoredObjects72CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT721 |
| ✓ | BbhColoredObjects73CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT2 074 |
| ✓ | BbhColoredObjects74CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT761 |
| ✓ | BbhColoredObjects75CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT1 258 |
| ✓ | BbhColoredObjects76CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT1 493 |
| ✓ | BbhColoredObjects77CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT391 |
| ✓ | BbhColoredObjects78CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 744 |
| ✓ | BbhColoredObjects79CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT961 |
| ✓ | BbhColoredObjects80CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT580 |
| ✓ | BbhColoredObjects81CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT890 |
| ✓ | BbhColoredObjects82CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT477 |
| ✓ | BbhColoredObjects83CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT184 |
| ✓ | BbhColoredObjects84CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT991 |
| ✓ | BbhColoredObjects85CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT620 |
| ✓ | BbhColoredObjects86CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT930 |
| ✓ | BbhColoredObjects87CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT804 |
| ✓ | BbhColoredObjects88CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT1 461 |
| ✓ | BbhColoredObjects89CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT740 |
| ✓ | BbhColoredObjects90CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT386 |
| ✓ | BbhColoredObjects91CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT467 |
| ✓ | BbhColoredObjects92CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT771 |
| ✓ | BbhColoredObjects93CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT664 |
| ✓ | BbhColoredObjects94CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 013 |
Reasoning - BBH reasoning about colored objects (set 4)
32 passed |
100.0% |
IN34 048 OUT23 419 | |
| ✓ | BbhReasoningAboutColoredObjects100CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 070 |
| ✓ | BbhReasoningAboutColoredObjects101CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT344 |
| ✓ | BbhReasoningAboutColoredObjects102CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT704 |
| ✓ | BbhReasoningAboutColoredObjects103CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT877 |
| ✓ | BbhReasoningAboutColoredObjects104CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT1 521 |
| ✓ | BbhReasoningAboutColoredObjects105CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 074 |
| ✓ | BbhReasoningAboutColoredObjects106CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT1 421 |
| ✓ | BbhReasoningAboutColoredObjects107CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT629 |
| ✓ | BbhReasoningAboutColoredObjects108CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT624 |
| ✓ | BbhReasoningAboutColoredObjects109CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT840 |
| ✓ | BbhReasoningAboutColoredObjects110CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT456 |
| ✓ | BbhReasoningAboutColoredObjects111CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT452 |
| ✓ | BbhReasoningAboutColoredObjects112CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT203 |
| ✓ | BbhReasoningAboutColoredObjects113CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT511 |
| ✓ | BbhReasoningAboutColoredObjects114CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 062 |
| ✓ | BbhReasoningAboutColoredObjects115CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT352 |
| ✓ | BbhReasoningAboutColoredObjects116CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT261 |
| ✓ | BbhReasoningAboutColoredObjects117CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT239 |
| ✓ | BbhReasoningAboutColoredObjects118CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 027 |
| ✓ | BbhReasoningAboutColoredObjects119CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT743 |
| ✓ | BbhReasoningAboutColoredObjects120CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT848 |
| ✓ | BbhReasoningAboutColoredObjects121CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT295 |
| ✓ | BbhReasoningAboutColoredObjects122CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT1 272 |
| ✓ | BbhReasoningAboutColoredObjects123CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT689 |
| ✓ | BbhReasoningAboutColoredObjects124CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT705 |
| ✓ | BbhReasoningAboutColoredObjects125CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT878 |
| ✓ | BbhReasoningAboutColoredObjects126CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT745 |
| ✓ | BbhReasoningAboutColoredObjects95CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT489 |
| ✓ | BbhReasoningAboutColoredObjects96CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT738 |
| ✓ | BbhReasoningAboutColoredObjects97CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 227 |
| ✓ | BbhReasoningAboutColoredObjects98CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT645 |
| ✓ | BbhReasoningAboutColoredObjects99CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT478 |
Reasoning - BBH reasoning about colored objects (set 5)
32 passed |
100.0% |
IN33 768 OUT30 502 | |
| ✓ | BbhReasoningAboutColoredObjects127CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT855 |
| ✓ | BbhReasoningAboutColoredObjects128CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT653 |
| ✓ | BbhReasoningAboutColoredObjects129CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT632 |
| ✓ | BbhReasoningAboutColoredObjects130CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT414 |
| ✓ | BbhReasoningAboutColoredObjects131CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT2 227 |
| ✓ | BbhReasoningAboutColoredObjects132CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 072 |
| ✓ | BbhReasoningAboutColoredObjects133CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 075 |
| ✓ | BbhReasoningAboutColoredObjects134CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT2 957 |
| ✓ | BbhReasoningAboutColoredObjects135CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT414 |
| ✓ | BbhReasoningAboutColoredObjects136CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 546 |
| ✓ | BbhReasoningAboutColoredObjects137CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT354 |
| ✓ | BbhReasoningAboutColoredObjects138CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT2 126 |
| ✓ | BbhReasoningAboutColoredObjects139CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT567 |
| ✓ | BbhReasoningAboutColoredObjects140CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT1 008 |
| ✓ | BbhReasoningAboutColoredObjects141CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT289 |
| ✓ | BbhReasoningAboutColoredObjects142CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT156 |
| ✓ | BbhReasoningAboutColoredObjects143CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT1 161 |
| ✓ | BbhReasoningAboutColoredObjects144CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 041 OUT1 378 |
| ✓ | BbhReasoningAboutColoredObjects145CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT1 281 |
| ✓ | BbhReasoningAboutColoredObjects146CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 015 |
| ✓ | BbhReasoningAboutColoredObjects147CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT1 216 |
| ✓ | BbhReasoningAboutColoredObjects148CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT822 |
| ✓ | BbhReasoningAboutColoredObjects149CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT474 |
| ✓ | BbhReasoningAboutColoredObjects150CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT1 003 |
| ✓ | BbhReasoningAboutColoredObjects151CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT653 |
| ✓ | BbhReasoningAboutColoredObjects152CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT247 |
| ✓ | BbhReasoningAboutColoredObjects153CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT846 |
| ✓ | BbhReasoningAboutColoredObjects154CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT754 |
| ✓ | BbhReasoningAboutColoredObjects155CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT928 |
| ✓ | BbhReasoningAboutColoredObjects156CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT974 |
| ✓ | BbhReasoningAboutColoredObjects157CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT571 |
| ✓ | BbhReasoningAboutColoredObjects158CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT834 |
Reasoning - BBH reasoning about colored objects (set 6)
32 passed |
100.0% |
IN34 011 OUT24 728 | |
| ✓ | BbhReasoningAboutColoredObjects159CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT528 |
| ✓ | BbhReasoningAboutColoredObjects160CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT1 142 |
| ✓ | BbhReasoningAboutColoredObjects161CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT673 |
| ✓ | BbhReasoningAboutColoredObjects162CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT694 |
| ✓ | BbhReasoningAboutColoredObjects163CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT288 |
| ✓ | BbhReasoningAboutColoredObjects164CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT523 |
| ✓ | BbhReasoningAboutColoredObjects165CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT609 |
| ✓ | BbhReasoningAboutColoredObjects166CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT728 |
| ✓ | BbhReasoningAboutColoredObjects167CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT881 |
| ✓ | BbhReasoningAboutColoredObjects168CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT982 |
| ✓ | BbhReasoningAboutColoredObjects169CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT478 |
| ✓ | BbhReasoningAboutColoredObjects170CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT192 |
| ✓ | BbhReasoningAboutColoredObjects171CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT777 |
| ✓ | BbhReasoningAboutColoredObjects172CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT531 |
| ✓ | BbhReasoningAboutColoredObjects173CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT162 |
| ✓ | BbhReasoningAboutColoredObjects174CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 034 OUT1 235 |
| ✓ | BbhReasoningAboutColoredObjects175CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT3 055 |
| ✓ | BbhReasoningAboutColoredObjects176CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT1 053 |
| ✓ | BbhReasoningAboutColoredObjects177CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT598 |
| ✓ | BbhReasoningAboutColoredObjects178CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT504 |
| ✓ | BbhReasoningAboutColoredObjects179CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT343 |
| ✓ | BbhReasoningAboutColoredObjects180CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT722 |
| ✓ | BbhReasoningAboutColoredObjects181CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT948 |
| ✓ | BbhReasoningAboutColoredObjects182CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 165 |
| ✓ | BbhReasoningAboutColoredObjects183CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT506 |
| ✓ | BbhReasoningAboutColoredObjects184CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT800 |
| ✓ | BbhReasoningAboutColoredObjects185CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT822 |
| ✓ | BbhReasoningAboutColoredObjects186CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT647 |
| ✓ | BbhReasoningAboutColoredObjects187CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT618 |
| ✓ | BbhReasoningAboutColoredObjects188CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT899 |
| ✓ | BbhReasoningAboutColoredObjects189CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT915 |
| ✓ | BbhReasoningAboutColoredObjects190CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT710 |
Reasoning - BBH reasoning about colored objects (set 7)
32 passed |
100.0% |
IN33 946 OUT23 692 | |
| ✓ | BbhReasoningAboutColoredObjects191CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT1 032 |
| ✓ | BbhReasoningAboutColoredObjects192CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 122 |
| ✓ | BbhReasoningAboutColoredObjects193CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT267 |
| ✓ | BbhReasoningAboutColoredObjects194CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT430 |
| ✓ | BbhReasoningAboutColoredObjects195CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT765 |
| ✓ | BbhReasoningAboutColoredObjects196CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT932 |
| ✓ | BbhReasoningAboutColoredObjects197CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT720 |
| ✓ | BbhReasoningAboutColoredObjects198CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT462 |
| ✓ | BbhReasoningAboutColoredObjects199CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT1 959 |
| ✓ | BbhReasoningAboutColoredObjects200CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT870 |
| ✓ | BbhReasoningAboutColoredObjects201CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 030 |
| ✓ | BbhReasoningAboutColoredObjects202CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT704 |
| ✓ | BbhReasoningAboutColoredObjects203CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT503 |
| ✓ | BbhReasoningAboutColoredObjects204CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT730 |
| ✓ | BbhReasoningAboutColoredObjects205CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT346 |
| ✓ | BbhReasoningAboutColoredObjects206CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT239 |
| ✓ | BbhReasoningAboutColoredObjects207CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT218 |
| ✓ | BbhReasoningAboutColoredObjects208CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT464 |
| ✓ | BbhReasoningAboutColoredObjects209CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT1 108 |
| ✓ | BbhReasoningAboutColoredObjects210CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT460 |
| ✓ | BbhReasoningAboutColoredObjects211CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT216 |
| ✓ | BbhReasoningAboutColoredObjects212CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT815 |
| ✓ | BbhReasoningAboutColoredObjects213CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 186 |
| ✓ | BbhReasoningAboutColoredObjects214CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 098 |
| ✓ | BbhReasoningAboutColoredObjects215CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 426 |
| ✓ | BbhReasoningAboutColoredObjects216CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT642 |
| ✓ | BbhReasoningAboutColoredObjects217CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT488 |
| ✓ | BbhReasoningAboutColoredObjects218CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT640 |
| ✓ | BbhReasoningAboutColoredObjects219CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT227 |
| ✓ | BbhReasoningAboutColoredObjects220CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT805 |
| ✓ | BbhReasoningAboutColoredObjects221CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT941 |
| ✓ | BbhReasoningAboutColoredObjects222CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT847 |
Reasoning - BBH reasoning about colored objects (set 8)
28 passed2 failed |
93.3% |
IN31 674 OUT32 242 | |
| ✓ | BbhReasoningAboutColoredObjects223CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 124 |
| ✓ | BbhReasoningAboutColoredObjects224CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT402 |
| ✓ | BbhReasoningAboutColoredObjects225CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT875 |
| ✓ | BbhReasoningAboutColoredObjects226CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT523 |
| ✓ | BbhReasoningAboutColoredObjects227CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT1 149 |
| ✓ | BbhReasoningAboutColoredObjects228CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT769 |
| ✓ | BbhReasoningAboutColoredObjects229CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT770 |
| ✓ | BbhReasoningAboutColoredObjects230CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT439 |
| ✓ | BbhReasoningAboutColoredObjects231CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT425 |
| × | BbhReasoningAboutColoredObjects232CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(F)")
Context: BbhReasoningAboutColoredObjects232CE: expected (F), got [e]
Values:
got = "e" |
Failed | IN1 050 OUT7 318 |
| ✓ | BbhReasoningAboutColoredObjects233CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT946 |
| ✓ | BbhReasoningAboutColoredObjects234CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT1 116 |
| ✓ | BbhReasoningAboutColoredObjects235CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT2 154 |
| ✓ | BbhReasoningAboutColoredObjects236CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT959 |
| ✓ | BbhReasoningAboutColoredObjects237CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT345 |
| ✓ | BbhReasoningAboutColoredObjects238CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT1 141 |
| ✓ | BbhReasoningAboutColoredObjects239CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT553 |
| ✓ | BbhReasoningAboutColoredObjects240CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT134 |
| ✓ | BbhReasoningAboutColoredObjects241CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT250 |
| ✓ | BbhReasoningAboutColoredObjects242CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT691 |
| ✓ | BbhReasoningAboutColoredObjects243CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT847 |
| ✓ | BbhReasoningAboutColoredObjects244CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT1 334 |
| ✓ | BbhReasoningAboutColoredObjects245CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT894 |
| ✓ | BbhReasoningAboutColoredObjects246CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT908 |
| ✓ | BbhReasoningAboutColoredObjects247CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT172 |
| ✓ | BbhReasoningAboutColoredObjects248CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT618 |
| ✓ | BbhReasoningAboutColoredObjects249CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT881 |
| × | BbhRuinNames10CEpick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames10CE: expected (A), got [d]
Values:
got = "d" |
Failed | IN996 OUT1 522 |
| ✓ | BbhRuinNames8CEpick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 316 |
| ✓ | BbhRuinNames9CEpick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 667 |
Reasoning - BBH ruin names (humorous edits) (set 1)
22 passed10 failed |
68.8% |
IN32 100 OUT46 494 | |
| ✓ | BbhRuinNames11CFWhich of the following is a humorous edit of this artist or movie name: 'the who!'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 467 |
| ✓ | BbhRuinNames12CFWhich of the following is a humorous edit of this artist or movie name: 'braveheart'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT828 |
| × | BbhRuinNames13CFWhich of the following is a humorous edit of this artist or movie name: 'the three stooges'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames13CF: expected (D), got [b]
Values:
got = "b" |
Failed | IN1 006 OUT769 |
| ✓ | BbhRuinNames14CFWhich of the following is a humorous edit of this artist or movie name: 'the deer hunter'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT400 |
| × | BbhRuinNames15CFWhich of the following is a humorous edit of this artist or movie name: 'boyz 2 men'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames15CF: expected (C), got [d]
Values:
got = "d" |
Failed | IN1 011 OUT2 457 |
| ✓ | BbhRuinNames16CFWhich of the following is a humorous edit of this artist or movie name: 'the beastie boys'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 132 |
| × | BbhRuinNames17CFWhich of the following is a humorous edit of this artist or movie name: 'joy division'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames17CF: expected (C), got [a]
Values:
got = "a" |
Failed | IN998 OUT1 602 |
| ✓ | BbhRuinNames18CFWhich of the following is a humorous edit of this artist or movie name: 'herbie hancock'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 982 |
| ✓ | BbhRuinNames19CFWhich of the following is a humorous edit of this artist or movie name: 'bon iver'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 084 |
| × | BbhRuinNames20CFWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames20CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN998 OUT1 085 |
| ✓ | BbhRuinNames21CFWhich of the following is a humorous edit of this artist or movie name: 'cinderella'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT875 |
| ✓ | BbhRuinNames22CFWhich of the following is a humorous edit of this artist or movie name: 'the italian job'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 469 |
| ✓ | BbhRuinNames23CFWhich of the following is a humorous edit of this artist or movie name: 'a few good men'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT866 |
| ✓ | BbhRuinNames24CFWhich of the following is a humorous edit of this artist or movie name: 'raiders of the lost ark'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT555 |
| ✓ | BbhRuinNames25CFWhich of the following is a humorous edit of this artist or movie name: 'little shop of horrors'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT4 162 |
| ✓ | BbhRuinNames26CFWhich of the following is a humorous edit of this artist or movie name: 'free willy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 417 |
| ✓ | BbhRuinNames27CFWhich of the following is a humorous edit of this artist or movie name: 'pixies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 249 |
| ✓ | BbhRuinNames28CFWhich of the following is a humorous edit of this artist or movie name: 'the princess and the frog'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 364 |
| ✓ | BbhRuinNames29CFWhich of the following is a humorous edit of this artist or movie name: 'foo fighters'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT417 |
| ✓ | BbhRuinNames30CFWhich of the following is a humorous edit of this artist or movie name: 'the godfather'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT825 |
| × | BbhRuinNames31CFWhich of the following is a humorous edit of this artist or movie name: 'pantera'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames31CF: expected (D), got [b]
Values:
got = "b" |
Failed | IN995 OUT2 978 |
| × | BbhRuinNames32CFWhich of the following is a humorous edit of this artist or movie name: 'top gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames32CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN995 OUT766 |
| ✓ | BbhRuinNames33CFWhich of the following is a humorous edit of this artist or movie name: 'die another day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT858 |
| ✓ | BbhRuinNames34CFWhich of the following is a humorous edit of this artist or movie name: 'the moody blues'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 339 |
| ✓ | BbhRuinNames35CFWhich of the following is a humorous edit of this artist or movie name: 'fall out boy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 092 |
| × | BbhRuinNames36CFWhich of the following is a humorous edit of this artist or movie name: 'love story'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames36CF: expected (A), got [c]
Values:
got = "c" |
Failed | IN998 OUT2 106 |
| × | BbhRuinNames37CFWhich of the following is a humorous edit of this artist or movie name: 'whitney houston'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames37CF: expected (A), got [b]
Values:
got = "b" |
Failed | IN1 008 OUT2 874 |
| ✓ | BbhRuinNames38CFWhich of the following is a humorous edit of this artist or movie name: 'bridge of spies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT925 |
| × | BbhRuinNames39CFWhich of the following is a humorous edit of this artist or movie name: 'breaking benjamin'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames39CF: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 005 OUT438 |
| × | BbhRuinNames40CFWhich of the following is a humorous edit of this artist or movie name: 'frank kappa'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames40CF: expected (C), got [d]
Values:
got = "d" |
Failed | IN999 OUT3 511 |
| ✓ | BbhRuinNames41CFWhich of the following is a humorous edit of this artist or movie name: 'jurassic park'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 337 |
| ✓ | BbhRuinNames42CFWhich of the following is a humorous edit of this artist or movie name: 'the band'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT2 265 |
Reasoning - BBH ruin names (humorous edits) (set 2)
26 passed6 failed |
81.3% |
IN32 136 OUT45 399 | |
| ✓ | BbhRuinNames43CFWhich of the following is a humorous edit of this artist or movie name: 'creedence clearwater revival'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 382 |
| × | BbhRuinNames44CFWhich of the following is a humorous edit of this artist or movie name: 'block 57'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames44CF: expected (D), got [a]
Values:
got = "a" |
Failed | IN1 008 OUT1 692 |
| ✓ | BbhRuinNames45CFWhich of the following is a humorous edit of this artist or movie name: 'get shorty'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT943 |
| × | BbhRuinNames46CFWhich of the following is a humorous edit of this artist or movie name: 'radiohead'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames46CF: expected (C), got [a]
Values:
got = "a" |
Failed | IN998 OUT1 868 |
| × | BbhRuinNames47CFWhich of the following is a humorous edit of this artist or movie name: 'mean girls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames47CF: expected (A), got [d]
Values:
got = "d" |
Failed | IN999 OUT1 457 |
| ✓ | BbhRuinNames48CFWhich of the following is a humorous edit of this artist or movie name: 'seven brides for seven brothers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT575 |
| ✓ | BbhRuinNames49CFWhich of the following is a humorous edit of this artist or movie name: 'the green mile'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 260 |
| ✓ | BbhRuinNames50CFWhich of the following is a humorous edit of this artist or movie name: 'cat on a hot tin roof'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT913 |
| ✓ | BbhRuinNames51CFWhich of the following is a humorous edit of this artist or movie name: 'new kids on the block'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT565 |
| ✓ | BbhRuinNames52CFWhich of the following is a humorous edit of this artist or movie name: 'superclunk'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 873 |
| ✓ | BbhRuinNames53CFWhich of the following is a humorous edit of this artist or movie name: 'my big fat greek wedding'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 010 |
| ✓ | BbhRuinNames54CFWhich of the following is a humorous edit of this artist or movie name: 'snakes on a plane'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 293 |
| × | BbhRuinNames55CFWhich of the following is a humorous edit of this artist or movie name: 'rocky'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames55CF: expected (A), got [d]
Values:
got = "d" |
Failed | IN995 OUT2 010 |
| ✓ | BbhRuinNames56CFWhich of the following is a humorous edit of this artist or movie name: 'pink floyd'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 491 |
| ✓ | BbhRuinNames57CFWhich of the following is a humorous edit of this artist or movie name: 'bad religion'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 349 |
| × | BbhRuinNames58CFWhich of the following is a humorous edit of this artist or movie name: 'the police'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames58CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN998 OUT2 110 |
| ✓ | BbhRuinNames59CFWhich of the following is a humorous edit of this artist or movie name: 'ice cube'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 021 |
| ✓ | BbhRuinNames60CFWhich of the following is a humorous edit of this artist or movie name: 'new order'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 500 |
| ✓ | BbhRuinNames61CFWhich of the following is a humorous edit of this artist or movie name: 'dolly dots'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 956 |
| ✓ | BbhRuinNames62CFWhich of the following is a humorous edit of this artist or movie name: 'die another day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT3 611 |
| ✓ | BbhRuinNames63CFWhich of the following is a humorous edit of this artist or movie name: 'mad max'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 212 |
| ✓ | BbhRuinNames64CFWhich of the following is a humorous edit of this artist or movie name: '8 mile'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 561 |
| ✓ | BbhRuinNames65CFWhich of the following is a humorous edit of this artist or movie name: 'i spit on your grave'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT980 |
| ✓ | BbhRuinNames66CFWhich of the following is a humorous edit of this artist or movie name: 'planet of the apes'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT486 |
| ✓ | BbhRuinNames67CFWhich of the following is a humorous edit of this artist or movie name: 'the rolling stones'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT557 |
| ✓ | BbhRuinNames68CFWhich of the following is a humorous edit of this artist or movie name: 'reservoir dogs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 132 |
| ✓ | BbhRuinNames69CFWhich of the following is a humorous edit of this artist or movie name: 'the men who stare at goats'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT960 |
| ✓ | BbhRuinNames70CFWhich of the following is a humorous edit of this artist or movie name: 'bridge of spies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT916 |
| ✓ | BbhRuinNames71CFWhich of the following is a humorous edit of this artist or movie name: 'last action hero'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 943 |
| ✓ | BbhRuinNames72CFWhich of the following is a humorous edit of this artist or movie name: 'the godfather'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 278 |
| ✓ | BbhRuinNames73CFWhich of the following is a humorous edit of this artist or movie name: 'the full monty'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 999 |
| × | BbhRuinNames74CFWhich of the following is a humorous edit of this artist or movie name: 'fast and furious'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames74CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN1 003 OUT2 496 |
Reasoning - BBH ruin names (humorous edits) (set 3)
27 passed5 failed |
84.4% |
IN32 159 OUT42 844 | |
| ✓ | BbhRuinNames100CFWhich of the following is a humorous edit of this artist or movie name: 'the battle of britain'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT653 |
| × | BbhRuinNames101CFWhich of the following is a humorous edit of this artist or movie name: 'dead man walking'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames101CF: expected (A), got [d]
Values:
got = "d" |
Failed | IN1 003 OUT1 559 |
| ✓ | BbhRuinNames102CFWhich of the following is a humorous edit of this artist or movie name: 'suicide squad'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT672 |
| ✓ | BbhRuinNames103CFWhich of the following is a humorous edit of this artist or movie name: 'the dark knight'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 500 |
| ✓ | BbhRuinNames104CFWhich of the following is a humorous edit of this artist or movie name: 'sleepless in seattle'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 604 |
| × | BbhRuinNames105CFWhich of the following is a humorous edit of this artist or movie name: 'pink floyd'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames105CF: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 005 OUT3 812 |
| ✓ | BbhRuinNames106CFWhich of the following is a humorous edit of this artist or movie name: 'the man with the golden gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT1 012 |
| ✓ | BbhRuinNames75CFWhich of the following is a humorous edit of this artist or movie name: 'skinny puppy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 237 |
| ✓ | BbhRuinNames76CFWhich of the following is a humorous edit of this artist or movie name: 'depeche mode'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 473 |
| ✓ | BbhRuinNames77CFWhich of the following is a humorous edit of this artist or movie name: 'sly and the family stone'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 272 |
| ✓ | BbhRuinNames78CFWhich of the following is a humorous edit of this artist or movie name: 'the departed'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 379 |
| ✓ | BbhRuinNames79CFWhich of the following is a humorous edit of this artist or movie name: 'blade runner'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 538 |
| ✓ | BbhRuinNames80CFWhich of the following is a humorous edit of this artist or movie name: 'the avengers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT937 |
| ✓ | BbhRuinNames81CFWhich of the following is a humorous edit of this artist or movie name: 'die hard'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 197 |
| ✓ | BbhRuinNames82CFWhich of the following is a humorous edit of this artist or movie name: 'weezer'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 160 |
| ✓ | BbhRuinNames83CFWhich of the following is a humorous edit of this artist or movie name: 'the longest day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT708 |
| × | BbhRuinNames84CFWhich of the following is a humorous edit of this artist or movie name: 'babs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames84CF: expected (A), got [d]
Values:
got = "d" |
Failed | IN994 OUT2 163 |
| ✓ | BbhRuinNames85CFWhich of the following is a humorous edit of this artist or movie name: 'moonlight'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT2 426 |
| ✓ | BbhRuinNames86CFWhich of the following is a humorous edit of this artist or movie name: 'animal farm'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT587 |
| ✓ | BbhRuinNames87CFWhich of the following is a humorous edit of this artist or movie name: 'seven brides for seven brothers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT856 |
| ✓ | BbhRuinNames88CFWhich of the following is a humorous edit of this artist or movie name: 'kiss'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 301 |
| ✓ | BbhRuinNames89CFWhich of the following is a humorous edit of this artist or movie name: 'the police'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT993 |
| ✓ | BbhRuinNames90CFWhich of the following is a humorous edit of this artist or movie name: 'en vogue'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 289 |
| ✓ | BbhRuinNames91CFWhich of the following is a humorous edit of this artist or movie name: 'top gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 124 |
| ✓ | BbhRuinNames92CFWhich of the following is a humorous edit of this artist or movie name: 'the girl with all the gifts'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT671 |
| ✓ | BbhRuinNames93CFWhich of the following is a humorous edit of this artist or movie name: 'the sisters of mercy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 652 |
| × | BbhRuinNames94CFWhich of the following is a humorous edit of this artist or movie name: 'the jam'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames94CF: expected (C), got [d]
Values:
got = "d" |
Failed | IN996 OUT1 375 |
| ✓ | BbhRuinNames95CFWhich of the following is a humorous edit of this artist or movie name: 'gone with the wind'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT2 162 |
| ✓ | BbhRuinNames96CFWhich of the following is a humorous edit of this artist or movie name: 'gone with the wind'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 069 |
| ✓ | BbhRuinNames97CFWhich of the following is a humorous edit of this artist or movie name: 'dream theater'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 254 |
| ✓ | BbhRuinNames98CFWhich of the following is a humorous edit of this artist or movie name: 'belle and sebastian'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT675 |
| × | BbhRuinNames99CFWhich of the following is a humorous edit of this artist or movie name: 'earth, wind, & fire'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("dearth, wind, & fire")
Context: BbhRuinNames99CF: expected dearth, wind, & fire, got [i]
Values:
got = "i" |
Failed | IN1 031 OUT1 534 |
Reasoning - BBH ruin names (humorous edits) (set 4)
31 passed1 failed |
96.9% |
IN32 143 OUT51 239 | |
| ✓ | BbhRuinNames107CFWhich of the following is a humorous edit of this artist or movie name: 'the mash'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 195 |
| ✓ | BbhRuinNames108CFWhich of the following is a humorous edit of this artist or movie name: 'missy elliott'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT630 |
| ✓ | BbhRuinNames109CFWhich of the following is a humorous edit of this artist or movie name: 'sleater-kinney'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 431 |
| ✓ | BbhRuinNames110CFWhich of the following is a humorous edit of this artist or movie name: 'dirty dancing'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT2 168 |
| ✓ | BbhRuinNames111CFWhich of the following is a humorous edit of this artist or movie name: 'jay and silent bob strike back'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT2 711 |
| ✓ | BbhRuinNames112CFWhich of the following is a humorous edit of this artist or movie name: 'a bridge too far'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 070 |
| ✓ | BbhRuinNames113CFWhich of the following is a humorous edit of this artist or movie name: 'the beatles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT998 |
| ✓ | BbhRuinNames114CFWhich of the following is a humorous edit of this artist or movie name: 'night club'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT2 279 |
| ✓ | BbhRuinNames115CFWhich of the following is a humorous edit of this artist or movie name: 'no time to die'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT674 |
| × | BbhRuinNames116CFWhich of the following is a humorous edit of this artist or movie name: 'batman begins'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames116CF: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 001 OUT1 577 |
| ✓ | BbhRuinNames117CFWhich of the following is a humorous edit of this artist or movie name: 'gold finger'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 106 |
| ✓ | BbhRuinNames118CFWhich of the following is a humorous edit of this artist or movie name: 'calamity jane'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 676 |
| ✓ | BbhRuinNames119CFWhich of the following is a humorous edit of this artist or movie name: 'saving private ryan'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 213 |
| ✓ | BbhRuinNames120CFWhich of the following is a humorous edit of this artist or movie name: 'the cult'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 242 |
| ✓ | BbhRuinNames121CFWhich of the following is a humorous edit of this artist or movie name: 'johnny got his gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT2 899 |
| ✓ | BbhRuinNames122CFWhich of the following is a humorous edit of this artist or movie name: 'counting crows'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 687 |
| ✓ | BbhRuinNames123CFWhich of the following is a humorous edit of this artist or movie name: 'lawrence of arabia'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 442 |
| ✓ | BbhRuinNames124CFWhich of the following is a humorous edit of this artist or movie name: 'jungle book'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT804 |
| ✓ | BbhRuinNames125CFWhich of the following is a humorous edit of this artist or movie name: 'paint your wagon'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT2 111 |
| ✓ | BbhRuinNames126CFWhich of the following is a humorous edit of this artist or movie name: 'blade'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT2 783 |
| ✓ | BbhRuinNames127CFWhich of the following is a humorous edit of this artist or movie name: 'the sisterhood of the traveling pants'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT1 204 |
| ✓ | BbhRuinNames128CFWhich of the following is a humorous edit of this artist or movie name: 'green day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 717 |
| ✓ | BbhRuinNames129CFWhich of the following is a humorous edit of this artist or movie name: 'the sting'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT738 |
| ✓ | BbhRuinNames130CFWhich of the following is a humorous edit of this artist or movie name: 'the artist'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 671 |
| ✓ | BbhRuinNames131CFWhich of the following is a humorous edit of this artist or movie name: 'the hope'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 139 |
| ✓ | BbhRuinNames132CFWhich of the following is a humorous edit of this artist or movie name: 'blade runner'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT3 988 |
| ✓ | BbhRuinNames133CFWhich of the following is a humorous edit of this artist or movie name: 'a view to a kill'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT713 |
| ✓ | BbhRuinNames134CFWhich of the following is a humorous edit of this artist or movie name: 'hope floats'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 197 |
| ✓ | BbhRuinNames135CFWhich of the following is a humorous edit of this artist or movie name: 'the apartment'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 652 |
| ✓ | BbhRuinNames136CFWhich of the following is a humorous edit of this artist or movie name: 'schindler's list'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT893 |
| ✓ | BbhRuinNames137CFWhich of the following is a humorous edit of this artist or movie name: 'beauty and the beast'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT917 |
| ✓ | BbhRuinNames138CFWhich of the following is a humorous edit of this artist or movie name: 'news of the world'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT714 |
Reasoning - BBH ruin names (humorous edits) (set 5)
27 passed3 failed |
90.0% |
IN30 139 OUT38 883 | |
| ✓ | BbhRuinNames139CFWhich of the following is a humorous edit of this artist or movie name: 'the guns of navarone'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT616 |
| ✓ | BbhRuinNames140CFWhich of the following is a humorous edit of this artist or movie name: 'the beatles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 113 |
| ✓ | BbhRuinNames141CFWhich of the following is a humorous edit of this artist or movie name: 'the godfather'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT828 |
| ✓ | BbhRuinNames142CFWhich of the following is a humorous edit of this artist or movie name: 'hank williams'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT609 |
| ✓ | BbhRuinNames143CFWhich of the following is a humorous edit of this artist or movie name: 'black sabbath'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 644 |
| × | BbhRuinNames144CFWhich of the following is a humorous edit of this artist or movie name: 'rita, sue and bob too'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("rita, sue and bob poo")
Context: BbhRuinNames144CF: expected rita, sue and bob poo, got [d]
Values:
got = "d" |
Failed | IN1 025 OUT1 172 |
| ✓ | BbhRuinNames145CFWhich of the following is a humorous edit of this artist or movie name: 'jethro tull'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 184 |
| ✓ | BbhRuinNames146CFWhich of the following is a humorous edit of this artist or movie name: 'the sound of music'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT722 |
| ✓ | BbhRuinNames147CFWhich of the following is a humorous edit of this artist or movie name: 'dream theater'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 592 |
| ✓ | BbhRuinNames148CFWhich of the following is a humorous edit of this artist or movie name: 'my fair lady'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 957 |
| ✓ | BbhRuinNames149CFWhich of the following is a humorous edit of this artist or movie name: 'offspring'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 735 |
| × | BbhRuinNames150CFWhich of the following is a humorous edit of this artist or movie name: 'the longest day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames150CF: expected (A), got [b]
Values:
got = "b" |
Failed | IN1 004 OUT2 820 |
| ✓ | BbhRuinNames151CFWhich of the following is a humorous edit of this artist or movie name: 'apocalypse now'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 033 |
| ✓ | BbhRuinNames152CFWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT2 411 |
| ✓ | BbhRuinNames153CFWhich of the following is a humorous edit of this artist or movie name: 'big country'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 889 |
| ✓ | BbhRuinNames154CFWhich of the following is a humorous edit of this artist or movie name: '12 angry men'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT688 |
| ✓ | BbhRuinNames155CFWhich of the following is a humorous edit of this artist or movie name: 'in search of a midnight kiss'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT703 |
| ✓ | BbhRuinNames156CFWhich of the following is a humorous edit of this artist or movie name: 'men in black'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT624 |
| ✓ | BbhRuinNames157CFWhich of the following is a humorous edit of this artist or movie name: 'the beatles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT396 |
| ✓ | BbhRuinNames158CFWhich of the following is a humorous edit of this artist or movie name: 'raging bull'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 316 |
| ✓ | BbhRuinNames159CFWhich of the following is a humorous edit of this artist or movie name: 'bad brains'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 442 |
| ✓ | BbhRuinNames160CFWhich of the following is a humorous edit of this artist or movie name: 'thin lizzy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 026 |
| ✓ | BbhRuinNames161CFWhich of the following is a humorous edit of this artist or movie name: 'ben folds five'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 220 |
| ✓ | BbhRuinNames162CFWhich of the following is a humorous edit of this artist or movie name: 'citizen kane'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT828 |
| ✓ | BbhRuinNames163CFWhich of the following is a humorous edit of this artist or movie name: 'dave matthews band'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 153 |
| ✓ | BbhRuinNames164CFWhich of the following is a humorous edit of this artist or movie name: 'the sum of all fears'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 116 |
| ✓ | BbhRuinNames165CFWhich of the following is a humorous edit of this artist or movie name: 'city hall'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 098 |
| × | BbhRuinNames166CFWhich of the following is a humorous edit of this artist or movie name: 'the abyss'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames166CF: expected (A), got [c]
Values:
got = "c" |
Failed | IN999 OUT1 459 |
| ✓ | BbhRuinNames167CFWhich of the following is a humorous edit of this artist or movie name: 'mank'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT2 381 |
| ✓ | BbhRuinNames168CFWhich of the following is a humorous edit of this artist or movie name: 'sex pistols'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 108 |
Reasoning - BBH ruin names (humorous edits) (set 6)
37 passed4 failed |
90.2% |
IN41 193 OUT51 645 | |
| × | BbhRuinNames169CGWhich of the following is a humorous edit of this artist or movie name: 'life of brian'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames169CG: expected (D), got [b]
Values:
got = "b" |
Failed | IN1 006 OUT2 030 |
| ✓ | BbhRuinNames170CGWhich of the following is a humorous edit of this artist or movie name: 'black panther'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 907 |
| ✓ | BbhRuinNames171CGWhich of the following is a humorous edit of this artist or movie name: 'goldfinger'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 536 |
| ✓ | BbhRuinNames172CGWhich of the following is a humorous edit of this artist or movie name: 'foo fighters'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 471 |
| ✓ | BbhRuinNames173CGWhich of the following is a humorous edit of this artist or movie name: 'air supply'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT914 |
| ✓ | BbhRuinNames174CGWhich of the following is a humorous edit of this artist or movie name: 'diana ross and the supremes'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT1 371 |
| ✓ | BbhRuinNames175CGWhich of the following is a humorous edit of this artist or movie name: 'brokeback mountain'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 016 |
| ✓ | BbhRuinNames176CGWhich of the following is a humorous edit of this artist or movie name: 'godzilla vs. kong'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 341 |
| ✓ | BbhRuinNames177CGWhich of the following is a humorous edit of this artist or movie name: 'the who!'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT976 |
| ✓ | BbhRuinNames178CGWhich of the following is a humorous edit of this artist or movie name: 'the green mile'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 027 |
| ✓ | BbhRuinNames179CGWhich of the following is a humorous edit of this artist or movie name: 'the dark knight'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 067 |
| ✓ | BbhRuinNames180CGWhich of the following is a humorous edit of this artist or movie name: 'live and let die'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT557 |
| × | BbhRuinNames181CGWhich of the following is a humorous edit of this artist or movie name: 'midnight cowboy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames181CG: expected (B), got [d]
Values:
got = "d" |
Failed | IN1 004 OUT2 424 |
| ✓ | BbhRuinNames182CGWhich of the following is a humorous edit of this artist or movie name: 'snow patrol'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT818 |
| ✓ | BbhRuinNames183CGWhich of the following is a humorous edit of this artist or movie name: 'the bangles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT2 157 |
| ✓ | BbhRuinNames184CGWhich of the following is a humorous edit of this artist or movie name: 'state champs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT699 |
| ✓ | BbhRuinNames185CGWhich of the following is a humorous edit of this artist or movie name: 'the secret life of pets'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT977 |
| ✓ | BbhRuinNames186CGWhich of the following is a humorous edit of this artist or movie name: 'full metal jacket'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 477 |
| ✓ | BbhRuinNames187CGWhich of the following is a humorous edit of this artist or movie name: 'friday the 13th'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT1 566 |
| ✓ | BbhRuinNames188CGWhich of the following is a humorous edit of this artist or movie name: 'the bay city rollers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT823 |
| ✓ | BbhRuinNames189CGWhich of the following is a humorous edit of this artist or movie name: 'blue'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 113 |
| ✓ | BbhRuinNames190CGWhich of the following is a humorous edit of this artist or movie name: 'the gap band'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT513 |
| ✓ | BbhRuinNames191CGWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT814 |
| ✓ | BbhRuinNames192CGWhich of the following is a humorous edit of this artist or movie name: 'the fall of the house of usher'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT801 |
| ✓ | BbhRuinNames193CGWhich of the following is a humorous edit of this artist or movie name: 'old joy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 995 |
| ✓ | BbhRuinNames194CGWhich of the following is a humorous edit of this artist or movie name: 'run dmc'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT2 607 |
| ✓ | BbhRuinNames195CGWhich of the following is a humorous edit of this artist or movie name: 'the grapes of wrath'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT679 |
| ✓ | BbhRuinNames196CGWhich of the following is a humorous edit of this artist or movie name: 'drag me to hell'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT453 |
| ✓ | BbhRuinNames197CGWhich of the following is a humorous edit of this artist or movie name: 'talking heads'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 221 |
| ✓ | BbhRuinNames198CGWhich of the following is a humorous edit of this artist or movie name: 'paul oakenfold'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 819 |
| ✓ | BbhRuinNames199CGWhich of the following is a humorous edit of this artist or movie name: 'brief encounter'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 858 |
| ✓ | BbhRuinNames200CGWhich of the following is a humorous edit of this artist or movie name: 'a tribe called quest'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT693 |
| × | BbhRuinNames201CGWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames201CG: expected (C), got [a]
Values:
got = "a" |
Failed | IN996 OUT1 504 |
| ✓ | BbhRuinNames202CGWhich of the following is a humorous edit of this artist or movie name: 'the wizard of oz'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT763 |
| ✓ | BbhRuinNames203CGWhich of the following is a humorous edit of this artist or movie name: 'run dmc'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT808 |
| × | BbhRuinNames204CGWhich of the following is a humorous edit of this artist or movie name: 'the third man'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames204CG: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 002 OUT1 951 |
| ✓ | BbhRuinNames205CGWhich of the following is a humorous edit of this artist or movie name: 'cat on a hot tin roof'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 368 |
| ✓ | BbhRuinNames206CGWhich of the following is a humorous edit of this artist or movie name: 'notting hill'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 331 |
| ✓ | BbhRuinNames207CGWhich of the following is a humorous edit of this artist or movie name: 'good will hunting'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 216 |
| ✓ | BbhRuinNames208CGWhich of the following is a humorous edit of this artist or movie name: 'foo fighters'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT854 |
| ✓ | BbhRuinNames209CGWhich of the following is a humorous edit of this artist or movie name: 'goo goo dolls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 130 |
Reasoning - BBH ruin names (humorous edits) (set 7)
37 passed3 failed |
92.5% |
IN40 207 OUT53 168 | |
| ✓ | BbhRuinNames210CGWhich of the following is a humorous edit of this artist or movie name: 'soundgarden'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 607 |
| ✓ | BbhRuinNames211CGWhich of the following is a humorous edit of this artist or movie name: 'peter and the test tube babies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 065 |
| ✓ | BbhRuinNames212CGWhich of the following is a humorous edit of this artist or movie name: 'blur'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 793 |
| ✓ | BbhRuinNames213CGWhich of the following is a humorous edit of this artist or movie name: 'black sabbath'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 177 |
| ✓ | BbhRuinNames214CGWhich of the following is a humorous edit of this artist or movie name: 'trolls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 649 |
| ✓ | BbhRuinNames215CGWhich of the following is a humorous edit of this artist or movie name: 'bad company'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT943 |
| ✓ | BbhRuinNames216CGWhich of the following is a humorous edit of this artist or movie name: '3:10 to yuma'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT3 560 |
| ✓ | BbhRuinNames217CGWhich of the following is a humorous edit of this artist or movie name: 'herman's hermits'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 054 |
| ✓ | BbhRuinNames218CGWhich of the following is a humorous edit of this artist or movie name: 'night of the demons'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT604 |
| ✓ | BbhRuinNames219CGWhich of the following is a humorous edit of this artist or movie name: 'the rolling stones'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT722 |
| ✓ | BbhRuinNames220CGWhich of the following is a humorous edit of this artist or movie name: 'blur'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 533 |
| ✓ | BbhRuinNames221CGWhich of the following is a humorous edit of this artist or movie name: 'a matter of life and death'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT571 |
| ✓ | BbhRuinNames222CGWhich of the following is a humorous edit of this artist or movie name: 'singin' in the rain'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT1 456 |
| ✓ | BbhRuinNames223CGWhich of the following is a humorous edit of this artist or movie name: 'the man with two brains'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT445 |
| ✓ | BbhRuinNames224CGWhich of the following is a humorous edit of this artist or movie name: 'born free'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 519 |
| ✓ | BbhRuinNames225CGWhich of the following is a humorous edit of this artist or movie name: 'the moody blues'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 255 |
| ✓ | BbhRuinNames226CGWhich of the following is a humorous edit of this artist or movie name: 'jojo rabbit'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT756 |
| ✓ | BbhRuinNames227CGWhich of the following is a humorous edit of this artist or movie name: 'reservoir dogs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 059 |
| ✓ | BbhRuinNames228CGWhich of the following is a humorous edit of this artist or movie name: 'thunder road'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT682 |
| ✓ | BbhRuinNames229CGWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 066 |
| ✓ | BbhRuinNames230CGWhich of the following is a humorous edit of this artist or movie name: 'deadpool'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT940 |
| ✓ | BbhRuinNames231CGWhich of the following is a humorous edit of this artist or movie name: 'the lady vanishes'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 170 |
| ✓ | BbhRuinNames232CGWhich of the following is a humorous edit of this artist or movie name: 'steely dan'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 408 |
| ✓ | BbhRuinNames233CGWhich of the following is a humorous edit of this artist or movie name: 'the men who stare at goats'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 766 |
| ✓ | BbhRuinNames234CGWhich of the following is a humorous edit of this artist or movie name: 'the stone roses'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 705 |
| ✓ | BbhRuinNames235CGWhich of the following is a humorous edit of this artist or movie name: 'nine inch nails'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT833 |
| ✓ | BbhRuinNames236CGWhich of the following is a humorous edit of this artist or movie name: 'billy idol'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 094 |
| ✓ | BbhRuinNames237CGWhich of the following is a humorous edit of this artist or movie name: 'schindler's list'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 343 |
| × | BbhRuinNames238CGWhich of the following is a humorous edit of this artist or movie name: 'sex pistols'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames238CG: expected (A), got [d]
Values:
got = "d" |
Failed | IN1 001 OUT4 527 |
| ✓ | BbhRuinNames239CGWhich of the following is a humorous edit of this artist or movie name: 'gone with the wind'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT524 |
| × | BbhRuinNames240CGWhich of the following is a humorous edit of this artist or movie name: 'it's a wonderful life'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames240CG: expected (A), got [b]
Values:
got = "b" |
Failed | IN1 014 OUT1 907 |
| ✓ | BbhRuinNames241CGWhich of the following is a humorous edit of this artist or movie name: 'guns n' roses'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 235 |
| ✓ | BbhRuinNames242CGWhich of the following is a humorous edit of this artist or movie name: 'witness'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 060 |
| × | BbhRuinNames243CGWhich of the following is a humorous edit of this artist or movie name: 'killing joke'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames243CG: expected (A), got [c]
Values:
got = "c" |
Failed | IN998 OUT1 695 |
| ✓ | BbhRuinNames244CGWhich of the following is a humorous edit of this artist or movie name: 'panic! at the disco'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 529 |
| ✓ | BbhRuinNames245CGWhich of the following is a humorous edit of this artist or movie name: 'spice girls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 170 |
| ✓ | BbhRuinNames246CGWhich of the following is a humorous edit of this artist or movie name: 'the police'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT869 |
| ✓ | BbhRuinNames247CGWhich of the following is a humorous edit of this artist or movie name: 'spandau ballet'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 021 |
| ✓ | BbhRuinNames248CGWhich of the following is a humorous edit of this artist or movie name: 'toto'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 803 |
| ✓ | BbhRuinNames249CGWhich of the following is a humorous edit of this artist or movie name: 'system of a down'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 053 |
Reasoning - BBH snarks - detect sarcasm (set 1)
27 passed5 failed |
84.4% |
IN33 394 OUT32 571 | |
| × | BbhSalientTranslation243CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSalientTranslation243CI: expected (A), got [e]
Values:
got = "e" |
Failed | IN1 202 OUT1 243 |
| ✓ | BbhSalientTranslation244CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 178 OUT2 147 |
| × | BbhSalientTranslation245CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSalientTranslation245CI: expected (B), got [d]
Values:
got = "d" |
Failed | IN1 192 OUT2 168 |
| ✓ | BbhSalientTranslation246CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 235 OUT1 094 |
| ✓ | BbhSalientTranslation247CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 190 OUT715 |
| × | BbhSalientTranslation248CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(E)")
Context: BbhSalientTranslation248CI: expected (E), got [a]
Values:
got = "a" |
Failed | IN1 185 OUT1 874 |
| × | BbhSalientTranslation249CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhSalientTranslation249CI: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 232 OUT1 993 |
| ✓ | BbhSnarks10CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT843 |
| ✓ | BbhSnarks11CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT691 |
| ✓ | BbhSnarks12CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT627 |
| ✓ | BbhSnarks13CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT694 |
| ✓ | BbhSnarks14CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT673 |
| ✓ | BbhSnarks15CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT600 |
| ✓ | BbhSnarks16CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT604 |
| ✓ | BbhSnarks17CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 135 |
| ✓ | BbhSnarks18CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT752 |
| ✓ | BbhSnarks19CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT551 |
| ✓ | BbhSnarks20CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT1 422 |
| ✓ | BbhSnarks21CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT998 |
| × | BbhSnarks22CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks22CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN994 OUT1 330 |
| ✓ | BbhSnarks23CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT2 020 |
| ✓ | BbhSnarks24CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT455 |
| ✓ | BbhSnarks25CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 838 |
| ✓ | BbhSnarks26CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 173 |
| ✓ | BbhSnarks27CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT352 |
| ✓ | BbhSnarks28CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT779 |
| ✓ | BbhSnarks29CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT723 |
| ✓ | BbhSnarks30CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT502 |
| ✓ | BbhSnarks31CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT621 |
| ✓ | BbhSnarks32CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT880 |
| ✓ | BbhSnarks8CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT602 |
| ✓ | BbhSnarks9CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT472 |
Reasoning - BBH snarks - detect sarcasm (set 2)
27 passed5 failed |
84.4% |
IN32 148 OUT35 980 | |
| ✓ | BbhSnarks33CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT617 |
| ✓ | BbhSnarks34CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT698 |
| ✓ | BbhSnarks35CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT772 |
| ✓ | BbhSnarks36CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT481 |
| ✓ | BbhSnarks37CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 645 |
| ✓ | BbhSnarks38CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT735 |
| ✓ | BbhSnarks39CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT571 |
| ✓ | BbhSnarks40CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT3 275 |
| × | BbhSnarks41CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks41CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 009 OUT1 939 |
| ✓ | BbhSnarks42CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT1 188 |
| × | BbhSnarks43CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks43CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 004 OUT1 672 |
| ✓ | BbhSnarks44CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT1 422 |
| ✓ | BbhSnarks45CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 048 |
| ✓ | BbhSnarks46CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT604 |
| ✓ | BbhSnarks47CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 858 |
| ✓ | BbhSnarks48CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 001 |
| × | BbhSnarks49CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSnarks49CI: expected (A), got [b]
Values:
got = "b" |
Failed | IN988 OUT2 104 |
| ✓ | BbhSnarks50CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 017 |
| × | BbhSnarks51CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks51CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 006 OUT2 174 |
| ✓ | BbhSnarks52CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT757 |
| ✓ | BbhSnarks53CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 002 |
| ✓ | BbhSnarks54CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT317 |
| ✓ | BbhSnarks55CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT593 |
| ✓ | BbhSnarks56CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 259 |
| ✓ | BbhSnarks57CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT875 |
| × | BbhSnarks58CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks58CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN996 OUT470 |
| ✓ | BbhSnarks59CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 294 |
| ✓ | BbhSnarks60CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 612 |
| ✓ | BbhSnarks61CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT713 |
| ✓ | BbhSnarks62CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT1 234 |
| ✓ | BbhSnarks63CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT375 |
| ✓ | BbhSnarks64CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT658 |
Reasoning - BBH snarks - detect sarcasm (set 3)
26 passed4 failed2 skipped |
86.7% |
IN30 091 OUT29 348 | |
| ✓ | BbhSnarks65CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT599 |
| × | BbhSnarks66CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks66CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 006 OUT1 358 |
| ✓ | BbhSnarks67CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT397 |
| ✓ | BbhSnarks68CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT500 |
| ✓ | BbhSnarks69CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT1 510 |
| ✓ | BbhSnarks70CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT465 |
| × | BbhSnarks71CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSnarks71CI: expected (A), got [b]
Values:
got = "b" |
Failed | IN998 OUT987 |
| ✓ | BbhSnarks72CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 327 |
| ✓ | BbhSnarks73CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT452 |
| ✓ | BbhSnarks74CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT746 |
| ✓ | BbhSnarks75CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT617 |
| ✓ | BbhSnarks76CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT619 |
| × | BbhSnarks77CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks77CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 040 OUT1 158 |
| × | BbhSnarks78CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks78CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 014 OUT934 |
| ✓ | BbhSnarks79CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT647 |
| ✓ | BbhSnarks80CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT957 |
| ✓ | BbhSnarks81CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT2 595 |
| ✓ | BbhSnarks82CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT1 065 |
| ✓ | BbhSnarks83CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT478 |
| ✓ | BbhSnarks84CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 122 |
| ✓ | BbhSnarks85CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 554 |
| – | BbhSnarks86CIWhich statement is sarcastic?0 AI queries
0 tool callscategory:Reasoninglevel:Hardsource:BBH Skipped: non-deterministic answer: both options are identical |
Skipped | IN0 OUT0 |
| ✓ | BbhSnarks87CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT823 |
| – | BbhSnarks88CIWhich statement is sarcastic?0 AI queries
0 tool callscategory:Reasoninglevel:Hardsource:BBH Skipped: non-deterministic answer: malformed item: single truncated option |
Skipped | IN0 OUT0 |
| ✓ | BbhSnarks89CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT663 |
| ✓ | BbhSnarks90CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT575 |
| ✓ | BbhSnarks91CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT515 |
| ✓ | BbhSnarks92CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT699 |
| ✓ | BbhSnarks93CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT3 169 |
| ✓ | BbhSnarks94CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT938 |
| ✓ | BbhSnarks95CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT992 |
| ✓ | BbhSnarks96CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT887 |
Reasoning - BBH snarks - detect sarcasm (set 4)
30 passed2 failed |
93.8% |
IN31 917 OUT28 599 | |
| ✓ | BbhSnarks100CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT905 |
| ✓ | BbhSnarks101CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT548 |
| ✓ | BbhSnarks102CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT665 |
| ✓ | BbhSnarks103CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT878 |
| ✓ | BbhSnarks104CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT521 |
| ✓ | BbhSnarks105CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 830 |
| ✓ | BbhSnarks106CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 077 |
| ✓ | BbhSnarks107CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT1 299 |
| ✓ | BbhSnarks108CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT624 |
| ✓ | BbhSnarks109CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT557 |
| ✓ | BbhSnarks110CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT939 |
| ✓ | BbhSnarks111CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT577 |
| ✓ | BbhSnarks112CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 216 |
| ✓ | BbhSnarks113CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT826 |
| ✓ | BbhSnarks114CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT590 |
| ✓ | BbhSnarks115CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT625 |
| ✓ | BbhSnarks116CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT689 |
| ✓ | BbhSnarks117CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 211 |
| ✓ | BbhSnarks118CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT899 |
| ✓ | BbhSnarks119CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT962 |
| ✓ | BbhSnarks120CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT650 |
| ✓ | BbhSnarks121CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT724 |
| ✓ | BbhSnarks122CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT738 |
| ✓ | BbhSnarks123CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT623 |
| ✓ | BbhSnarks124CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 224 |
| × | BbhSnarks125CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks125CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 000 OUT802 |
| ✓ | BbhSnarks126CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT801 |
| ✓ | BbhSnarks127CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT643 |
| × | BbhSnarks128CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSnarks128CI: expected (A), got [b]
Values:
got = "b" |
Failed | IN998 OUT1 089 |
| ✓ | BbhSnarks97CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT1 140 |
| ✓ | BbhSnarks98CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 437 |
| ✓ | BbhSnarks99CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 290 |
Reasoning - BBH snarks - detect sarcasm (set 5)
29 passed1 failed |
96.7% |
IN30 055 OUT30 286 | |
| ✓ | BbhSnarks129CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT562 |
| ✓ | BbhSnarks130CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT450 |
| ✓ | BbhSnarks131CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT2 001 |
| ✓ | BbhSnarks132CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT447 |
| ✓ | BbhSnarks133CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT1 156 |
| ✓ | BbhSnarks134CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT407 |
| ✓ | BbhSnarks135CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 243 |
| ✓ | BbhSnarks136CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 060 |
| ✓ | BbhSnarks137CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT796 |
| ✓ | BbhSnarks138CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT795 |
| ✓ | BbhSnarks139CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT804 |
| ✓ | BbhSnarks140CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 018 |
| ✓ | BbhSnarks141CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT894 |
| ✓ | BbhSnarks142CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT1 049 |
| ✓ | BbhSnarks143CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 151 |
| × | BbhSnarks144CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks144CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN998 OUT1 932 |
| ✓ | BbhSnarks145CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 039 |
| ✓ | BbhSnarks146CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT641 |
| ✓ | BbhSnarks147CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT872 |
| ✓ | BbhSnarks148CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 207 |
| ✓ | BbhSnarks149CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT410 |
| ✓ | BbhSnarks150CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT936 |
| ✓ | BbhSnarks151CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT768 |
| ✓ | BbhSnarks152CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 727 |
| ✓ | BbhSnarks153CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 074 |
| ✓ | BbhSnarks154CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT1 430 |
| ✓ | BbhSnarks155CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 291 |
| ✓ | BbhSnarks156CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT685 |
| ✓ | BbhSnarks157CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 124 |
| ✓ | BbhSnarks158CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 317 |
Reasoning - BBH snarks - detect sarcasm (set 6)
28 passed4 failed |
87.5% |
IN31 595 OUT30 143 | |
| ✓ | BbhSnarks159CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT736 |
| ✓ | BbhSnarks160CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT380 |
| ✓ | BbhSnarks161CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT626 |
| ✓ | BbhSnarks162CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT570 |
| ✓ | BbhSnarks163CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT543 |
| ✓ | BbhSnarks164CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 190 |
| ✓ | BbhSnarks165CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT988 |
| ✓ | BbhSnarks166CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT806 |
| ✓ | BbhSnarks167CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT568 |
| ✓ | BbhSnarks168CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 330 |
| ✓ | BbhSnarks169CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 102 |
| ✓ | BbhSnarks170CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 982 |
| ✓ | BbhSnarks171CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 521 |
| ✓ | BbhSnarks172CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT505 |
| ✓ | BbhSnarks173CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT587 |
| ✓ | BbhSnarks174CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT580 |
| ✓ | BbhSnarks175CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT555 |
| ✓ | BbhSnarks176CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT726 |
| ✓ | BbhSnarks177CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT374 |
| ✓ | BbhSportsUnderstanding10CJIs the following sentence plausible? "Carson Wentz took to the ice."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN969 OUT616 |
| ✓ | BbhSportsUnderstanding11CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN972 OUT1 344 |
| ✓ | BbhSportsUnderstanding12CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN975 OUT1 761 |
| ✓ | BbhSportsUnderstanding13CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN970 OUT1 137 |
| ✓ | BbhSportsUnderstanding14CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN970 OUT1 007 |
| × | BbhSportsUnderstanding15CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding15CJ expected yes, got [no]
Values:
got = "no" |
Failed | IN973 OUT2 974 |
| ✓ | BbhSportsUnderstanding16CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN969 OUT684 |
| ✓ | BbhSportsUnderstanding17CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN968 OUT836 |
| ✓ | BbhSportsUnderstanding18CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN967 OUT898 |
| × | BbhSportsUnderstanding19CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding19CJ expected yes, got [no]
Values:
got = "no" |
Failed | IN974 OUT321 |
| ✓ | BbhSportsUnderstanding20CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN968 OUT782 |
| × | BbhSportsUnderstanding8CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding8CJ expected yes, got [no]
Values:
got = "no" |
Failed | IN973 OUT978 |
| × | BbhSportsUnderstanding9CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("no")
Context: BbhSportsUnderstanding9CJ expected no, got [yes]
Values:
got = "yes" |
Failed | IN969 OUT1 136 |
Reasoning - BBH temporal sequences (set 1)
30 passed2 failed |
93.8% |
IN35 127 OUT37 076 | |
| ✓ | BbhSportsUnderstanding243CKIs the following sentence plausible? "Blake Snell hit a single."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN967 OUT647 |
| × | BbhSportsUnderstanding244CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding244CK expected yes, got [false]
Values:
got = "false" |
Failed | IN974 OUT1 145 |
| ✓ | BbhSportsUnderstanding245CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN972 OUT461 |
| × | BbhSportsUnderstanding246CKIs the following sentence plausible? "Pepe converted the first down."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("no")
Context: BbhSportsUnderstanding246CK expected no, got [yes]
Values:
got = "yes" |
Failed | IN967 OUT1 434 |
| ✓ | BbhSportsUnderstanding247CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN975 OUT2 481 |
| ✓ | BbhSportsUnderstanding248CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN969 OUT996 |
| ✓ | BbhSportsUnderstanding249CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN970 OUT1 123 |
| ✓ | BbhTemporalSequences10CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT1 080 |
| ✓ | BbhTemporalSequences11CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT927 |
| ✓ | BbhTemporalSequences12CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 444 |
| ✓ | BbhTemporalSequences13CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 138 OUT1 103 |
| ✓ | BbhTemporalSequences14CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 484 |
| ✓ | BbhTemporalSequences15CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT1 262 |
| ✓ | BbhTemporalSequences16CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT1 141 |
| ✓ | BbhTemporalSequences17CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT1 108 |
| ✓ | BbhTemporalSequences18CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT1 698 |
| ✓ | BbhTemporalSequences19CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT730 |
| ✓ | BbhTemporalSequences20CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 283 |
| ✓ | BbhTemporalSequences21CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT995 |
| ✓ | BbhTemporalSequences22CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT1 451 |
| ✓ | BbhTemporalSequences23CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT1 194 |
| ✓ | BbhTemporalSequences24CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT958 |
| ✓ | BbhTemporalSequences25CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT1 094 |
| ✓ | BbhTemporalSequences26CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT1 099 |
| ✓ | BbhTemporalSequences27CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT1 025 |
| ✓ | BbhTemporalSequences28CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT999 |
| ✓ | BbhTemporalSequences29CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 231 |
| ✓ | BbhTemporalSequences30CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT882 |
| ✓ | BbhTemporalSequences31CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT978 |
| ✓ | BbhTemporalSequences32CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT1 028 |
| ✓ | BbhTemporalSequences8CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 687 |
| ✓ | BbhTemporalSequences9CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT908 |
Reasoning - BBH temporal sequences (set 2)
30 passed |
100.0% |
IN33 903 OUT34 318 | |
| ✓ | BbhTemporalSequences33CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT1 407 |
| ✓ | BbhTemporalSequences34CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 153 |
| ✓ | BbhTemporalSequences35CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 291 |
| ✓ | BbhTemporalSequences36CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT1 049 |
| ✓ | BbhTemporalSequences37CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 416 |
| ✓ | BbhTemporalSequences38CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT1 300 |
| ✓ | BbhTemporalSequences39CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 048 |
| ✓ | BbhTemporalSequences40CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 237 |
| ✓ | BbhTemporalSequences41CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT907 |
| ✓ | BbhTemporalSequences42CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT1 005 |
| ✓ | BbhTemporalSequences43CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT921 |
| ✓ | BbhTemporalSequences44CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT911 |
| ✓ | BbhTemporalSequences45CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT821 |
| ✓ | BbhTemporalSequences46CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT1 067 |
| ✓ | BbhTemporalSequences47CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT1 363 |
| ✓ | BbhTemporalSequences48CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 622 |
| ✓ | BbhTemporalSequences49CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT1 255 |
| ✓ | BbhTemporalSequences50CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT965 |
| ✓ | BbhTemporalSequences51CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT961 |
| ✓ | BbhTemporalSequences52CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT733 |
| ✓ | BbhTemporalSequences53CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 465 |
| ✓ | BbhTemporalSequences54CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT1 377 |
| ✓ | BbhTemporalSequences55CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT869 |
| ✓ | BbhTemporalSequences56CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT1 120 |
| ✓ | BbhTemporalSequences57CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT984 |
| ✓ | BbhTemporalSequences58CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT1 308 |
| ✓ | BbhTemporalSequences59CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT1 060 |
| ✓ | BbhTemporalSequences60CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT1 204 |
| ✓ | BbhTemporalSequences61CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT1 115 |
| ✓ | BbhTemporalSequences62CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 384 |
Reasoning - BBH temporal sequences (set 3)
32 passed |
100.0% |
IN36 111 OUT37 132 | |
| ✓ | BbhTemporalSequences63CLfind the time window when Samantha could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 110 |
| ✓ | BbhTemporalSequences64CLfind the time window when Betty could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 191 |
| ✓ | BbhTemporalSequences65CLfind the time window when William could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT1 119 |
| ✓ | BbhTemporalSequences66CLfind the time window when James could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT947 |
| ✓ | BbhTemporalSequences67CLfind the time window when Samantha could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT965 |
| ✓ | BbhTemporalSequences68CLfind the time window when William could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT1 230 |
| ✓ | BbhTemporalSequences69CLfind the time window when Kimberly could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT1 127 |
| ✓ | BbhTemporalSequences70CLfind the time window when Susan could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 091 |
| ✓ | BbhTemporalSequences71CLfind the time window when James could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT830 |
| ✓ | BbhTemporalSequences72CLfind the time window when Ashley could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 123 |
| ✓ | BbhTemporalSequences73CLfind the time window when Jennifer could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 032 |
| ✓ | BbhTemporalSequences74CLfind the time window when James could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 172 |
| ✓ | BbhTemporalSequences75CLfind the time window when Anthony could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 152 |
| ✓ | BbhTemporalSequences76CLfind the time window when Thomas could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 236 |
| ✓ | BbhTemporalSequences77CLfind the time window when Jennifer could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 384 |
| ✓ | BbhTemporalSequences78CLfind the time window when Sarah could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT771 |
| ✓ | BbhTemporalSequences79CLfind the time window when Susan could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 105 |
| ✓ | BbhTemporalSequences80CLfind the time window when William could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT1 266 |
| ✓ | BbhTemporalSequences81CLfind the time window when Jessica could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT998 |
| ✓ | BbhTemporalSequences82CLfind the time window when Kimberly could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT1 315 |
| ✓ | BbhTemporalSequences83CLfind the time window when Elizabeth could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT908 |
| ✓ | BbhTemporalSequences84CLfind the time window when James could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT2 189 |
| ✓ | BbhTemporalSequences85CLfind the time window when Michael could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT1 297 |
| ✓ | BbhTemporalSequences86CLfind the time window when Sarah could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT1 058 |
| ✓ | BbhTemporalSequences87CLfind the time window when Mark could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 413 |
| ✓ | BbhTemporalSequences88CLfind the time window when Sean could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT1 120 |
| ✓ | BbhTemporalSequences89CLfind the time window when James could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT1 874 |
| ✓ | BbhTemporalSequences90CLfind the time window when Jessica could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT898 |
| ✓ | BbhTemporalSequences91CLfind the time window when Tiffany could have gone to the library1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT751 |
| ✓ | BbhTemporalSequences92CLfind the time window when James could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT1 338 |
| ✓ | BbhTemporalSequences93CLfind the time window when Nancy could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT973 |
| ✓ | BbhTemporalSequences94CLfind the time window when Tiffany could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 149 |
Reasoning - BBH temporal sequences (set 4)
32 passed |
100.0% |
IN36 005 OUT35 320 | |
| ✓ | BbhTemporalSequences100CLfind the time window when Steven could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT960 |
| ✓ | BbhTemporalSequences101CLfind the time window when Jessica could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT1 310 |
| ✓ | BbhTemporalSequences102CLfind the time window when Jessica could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT956 |
| ✓ | BbhTemporalSequences103CLfind the time window when Emily could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT1 451 |
| ✓ | BbhTemporalSequences104CLfind the time window when Anthony could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT836 |
| ✓ | BbhTemporalSequences105CLfind the time window when Mark could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT1 200 |
| ✓ | BbhTemporalSequences106CLfind the time window when William could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 794 |
| ✓ | BbhTemporalSequences107CLfind the time window when Hannah could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT813 |
| ✓ | BbhTemporalSequences108CLfind the time window when William could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT864 |
| ✓ | BbhTemporalSequences109CLfind the time window when Nancy could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 004 |
| ✓ | BbhTemporalSequences110CLfind the time window when Andrew could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 038 |
| ✓ | BbhTemporalSequences111CLfind the time window when David could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT1 155 |
| ✓ | BbhTemporalSequences112CLfind the time window when Anthony could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT695 |
| ✓ | BbhTemporalSequences113CLfind the time window when Sarah could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT860 |
| ✓ | BbhTemporalSequences114CLfind the time window when Mary could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 166 |
| ✓ | BbhTemporalSequences115CLfind the time window when Mark could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 099 |
| ✓ | BbhTemporalSequences116CLfind the time window when William could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 088 |
| ✓ | BbhTemporalSequences117CLfind the time window when Steven could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT1 060 |
| ✓ | BbhTemporalSequences118CLfind the time window when Leslie could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 672 |
| ✓ | BbhTemporalSequences119CLfind the time window when Sarah could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT950 |
| ✓ | BbhTemporalSequences120CLfind the time window when Andrew could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT1 104 |
| ✓ | BbhTemporalSequences121CLfind the time window when David could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT1 084 |
| ✓ | BbhTemporalSequences122CLfind the time window when Jessica could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT1 229 |
| ✓ | BbhTemporalSequences123CLfind the time window when David could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 093 |
| ✓ | BbhTemporalSequences124CLfind the time window when Jessica could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT1 173 |
| ✓ | BbhTemporalSequences125CLfind the time window when Mary could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 046 |
| ✓ | BbhTemporalSequences126CLfind the time window when Michael could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT943 |
| ✓ | BbhTemporalSequences95CLfind the time window when Jennifer could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT923 |
| ✓ | BbhTemporalSequences96CLfind the time window when Sean could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 141 OUT1 696 |
| ✓ | BbhTemporalSequences97CLfind the time window when Samantha could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 126 |
| ✓ | BbhTemporalSequences98CLfind the time window when Ashley could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT862 |
| ✓ | BbhTemporalSequences99CLfind the time window when Sarah could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 137 OUT1 070 |
Reasoning - BBH temporal sequences (set 5)
32 passed |
100.0% |
IN36 149 OUT35 751 | |
| ✓ | BbhTemporalSequences127CLfind the time window when David could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 035 |
| ✓ | BbhTemporalSequences128CLfind the time window when Andrew could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT1 145 |
| ✓ | BbhTemporalSequences129CLfind the time window when Leslie could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT1 153 |
| ✓ | BbhTemporalSequences130CLfind the time window when Mary could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT1 074 |
| ✓ | BbhTemporalSequences131CLfind the time window when Ashley could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT788 |
| ✓ | BbhTemporalSequences132CLfind the time window when Samantha could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 021 |
| ✓ | BbhTemporalSequences133CLfind the time window when Elizabeth could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT1 161 |
| ✓ | BbhTemporalSequences134CLfind the time window when Sean could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT1 201 |
| ✓ | BbhTemporalSequences135CLfind the time window when Elizabeth could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT949 |
| ✓ | BbhTemporalSequences136CLfind the time window when Michael could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT980 |
| ✓ | BbhTemporalSequences137CLfind the time window when Andrew could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT943 |
| ✓ | BbhTemporalSequences138CLfind the time window when Nancy could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT934 |
| ✓ | BbhTemporalSequences139CLfind the time window when Jason could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 123 OUT1 226 |
| ✓ | BbhTemporalSequences140CLfind the time window when Anthony could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 320 |
| ✓ | BbhTemporalSequences141CLfind the time window when Leslie could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 081 |
| ✓ | BbhTemporalSequences142CLfind the time window when David could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT1 490 |
| ✓ | BbhTemporalSequences143CLfind the time window when Hannah could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT1 025 |
| ✓ | BbhTemporalSequences144CLfind the time window when Sarah could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT1 194 |
| ✓ | BbhTemporalSequences145CLfind the time window when Andrew could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT1 239 |
| ✓ | BbhTemporalSequences146CLfind the time window when William could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 154 |
| ✓ | BbhTemporalSequences147CLfind the time window when Elizabeth could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT983 |
| ✓ | BbhTemporalSequences148CLfind the time window when Sarah could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 237 |
| ✓ | BbhTemporalSequences149CLfind the time window when David could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT911 |
| ✓ | BbhTemporalSequences150CLfind the time window when Jason could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT980 |
| ✓ | BbhTemporalSequences151CLfind the time window when Anthony could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 136 |
| ✓ | BbhTemporalSequences152CLfind the time window when Sarah could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT1 089 |
| ✓ | BbhTemporalSequences153CLfind the time window when Hannah could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT1 377 |
| ✓ | BbhTemporalSequences154CLfind the time window when Mark could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 273 |
| ✓ | BbhTemporalSequences155CLfind the time window when Sarah could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT803 |
| ✓ | BbhTemporalSequences156CLfind the time window when James could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT1 292 |
| ✓ | BbhTemporalSequences157CLfind the time window when Emily could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 298 |
| ✓ | BbhTemporalSequences158CLfind the time window when David could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT1 259 |
Reasoning - BBH temporal sequences (set 6)
32 passed |
100.0% |
IN36 117 OUT35 726 | |
| ✓ | BbhTemporalSequences159CLfind the time window when Jennifer could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT1 022 |
| ✓ | BbhTemporalSequences160CLfind the time window when Ashley could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT686 |
| ✓ | BbhTemporalSequences161CLfind the time window when William could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 123 OUT1 000 |
| ✓ | BbhTemporalSequences162CLfind the time window when Jennifer could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT1 189 |
| ✓ | BbhTemporalSequences163CLfind the time window when Jennifer could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT1 094 |
| ✓ | BbhTemporalSequences164CLfind the time window when Jennifer could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT1 033 |
| ✓ | BbhTemporalSequences165CLfind the time window when Nancy could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT1 193 |
| ✓ | BbhTemporalSequences166CLfind the time window when Samantha could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 815 |
| ✓ | BbhTemporalSequences167CLfind the time window when Tiffany could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT925 |
| ✓ | BbhTemporalSequences168CLfind the time window when Leslie could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT1 166 |
| ✓ | BbhTemporalSequences169CLfind the time window when Hannah could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT1 336 |
| ✓ | BbhTemporalSequences170CLfind the time window when Anthony could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT1 207 |
| ✓ | BbhTemporalSequences171CLfind the time window when Hannah could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT1 713 |
| ✓ | BbhTemporalSequences172CLfind the time window when Leslie could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 154 |
| ✓ | BbhTemporalSequences173CLfind the time window when Anthony could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT599 |
| ✓ | BbhTemporalSequences174CLfind the time window when Thomas could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 283 |
| ✓ | BbhTemporalSequences175CLfind the time window when Jason could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 484 |
| ✓ | BbhTemporalSequences176CLfind the time window when Betty could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT871 |
| ✓ | BbhTemporalSequences177CLfind the time window when Ashley could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT578 |
| ✓ | BbhTemporalSequences178CLfind the time window when David could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT995 |
| ✓ | BbhTemporalSequences179CLfind the time window when Leslie could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 178 |
| ✓ | BbhTemporalSequences180CLfind the time window when Ashley could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT792 |
| ✓ | BbhTemporalSequences181CLfind the time window when Michael could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT907 |
| ✓ | BbhTemporalSequences182CLfind the time window when Jason could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 401 |
| ✓ | BbhTemporalSequences183CLfind the time window when Sean could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT1 539 |
| ✓ | BbhTemporalSequences184CLfind the time window when Sarah could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 215 |
| ✓ | BbhTemporalSequences185CLfind the time window when Andrew could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 279 |
| ✓ | BbhTemporalSequences186CLfind the time window when James could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT983 |
| ✓ | BbhTemporalSequences187CLfind the time window when Ashley could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT1 327 |
| ✓ | BbhTemporalSequences188CLfind the time window when Ashley could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT707 |
| ✓ | BbhTemporalSequences189CLfind the time window when Sarah could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT895 |
| ✓ | BbhTemporalSequences190CLfind the time window when Michael could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT1 160 |
Reasoning - BBH temporal sequences (set 7)
30 passed |
100.0% |
IN33 806 OUT33 360 | |
| ✓ | BbhTemporalSequences191CLfind the time window when Anthony could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT949 |
| ✓ | BbhTemporalSequences192CLfind the time window when William could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT912 |
| ✓ | BbhTemporalSequences193CLfind the time window when Susan could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT1 293 |
| ✓ | BbhTemporalSequences194CLfind the time window when Sarah could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 061 |
| ✓ | BbhTemporalSequences195CLfind the time window when Emily could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT672 |
| ✓ | BbhTemporalSequences196CLfind the time window when Emily could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT1 706 |
| ✓ | BbhTemporalSequences197CLfind the time window when Sean could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 193 |
| ✓ | BbhTemporalSequences198CLfind the time window when Elizabeth could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 141 OUT639 |
| ✓ | BbhTemporalSequences199CLfind the time window when William could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT935 |
| ✓ | BbhTemporalSequences200CLfind the time window when Thomas could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT1 436 |
| ✓ | BbhTemporalSequences201CLfind the time window when Susan could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT1 031 |
| ✓ | BbhTemporalSequences202CLfind the time window when Kimberly could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 158 OUT1 520 |
| ✓ | BbhTemporalSequences203CLfind the time window when Leslie could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 042 |
| ✓ | BbhTemporalSequences204CLfind the time window when William could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 161 |
| ✓ | BbhTemporalSequences205CLfind the time window when Andrew could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT1 028 |
| ✓ | BbhTemporalSequences206CLfind the time window when Jennifer could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 241 |
| ✓ | BbhTemporalSequences207CLfind the time window when Susan could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT980 |
| ✓ | BbhTemporalSequences208CLfind the time window when James could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT1 481 |
| ✓ | BbhTemporalSequences209CLfind the time window when Tiffany could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT963 |
| ✓ | BbhTemporalSequences210CLfind the time window when Andrew could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 035 |
| ✓ | BbhTemporalSequences211CLfind the time window when Thomas could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT942 |
| ✓ | BbhTemporalSequences212CLfind the time window when David could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT1 678 |
| ✓ | BbhTemporalSequences213CLfind the time window when Elizabeth could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 191 |
| ✓ | BbhTemporalSequences214CLfind the time window when Jason could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT1 056 |
| ✓ | BbhTemporalSequences215CLfind the time window when Sarah could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT829 |
| ✓ | BbhTemporalSequences216CLfind the time window when William could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 060 |
| ✓ | BbhTemporalSequences217CLfind the time window when Ashley could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT853 |
| ✓ | BbhTemporalSequences218CLfind the time window when William could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT1 234 |
| ✓ | BbhTemporalSequences219CLfind the time window when Sarah could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT1 303 |
| ✓ | BbhTemporalSequences220CLfind the time window when Nancy could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT936 |
Reasoning - BBH temporal sequences (set 8)
29 passed |
100.0% |
IN32 959 OUT32 504 | |
| ✓ | BbhTemporalSequences221CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT1 071 |
| ✓ | BbhTemporalSequences222CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT1 031 |
| ✓ | BbhTemporalSequences223CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT1 180 |
| ✓ | BbhTemporalSequences224CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT947 |
| ✓ | BbhTemporalSequences225CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 195 |
| ✓ | BbhTemporalSequences226CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT1 865 |
| ✓ | BbhTemporalSequences227CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT715 |
| ✓ | BbhTemporalSequences228CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT934 |
| ✓ | BbhTemporalSequences229CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 121 OUT965 |
| ✓ | BbhTemporalSequences230CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT954 |
| ✓ | BbhTemporalSequences231CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 069 |
| ✓ | BbhTemporalSequences232CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT1 396 |
| ✓ | BbhTemporalSequences233CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 144 |
| ✓ | BbhTemporalSequences234CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 033 |
| ✓ | BbhTemporalSequences235CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 053 |
| ✓ | BbhTemporalSequences236CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT901 |
| ✓ | BbhTemporalSequences237CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT1 355 |
| ✓ | BbhTemporalSequences238CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT1 471 |
| ✓ | BbhTemporalSequences239CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 468 |
| ✓ | BbhTemporalSequences240CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 138 |
| ✓ | BbhTemporalSequences241CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT1 256 |
| ✓ | BbhTemporalSequences242CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT852 |
| ✓ | BbhTemporalSequences243CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT1 147 |
| ✓ | BbhTemporalSequences244CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT1 064 |
| ✓ | BbhTemporalSequences245CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT985 |
| ✓ | BbhTemporalSequences246CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT468 |
| ✓ | BbhTemporalSequences247CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 116 |
| ✓ | BbhTemporalSequences248CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT1 623 |
| ✓ | BbhTemporalSequences249CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 108 |
Reasoning - BBH tracking shuffled objects (3) (set 1)
30 passed |
100.0% |
IN31 987 OUT25 816 | |
| ✓ | BbhTrackingThree10CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT860 |
| ✓ | BbhTrackingThree11CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT995 |
| ✓ | BbhTrackingThree12CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 390 |
| ✓ | BbhTrackingThree13CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT891 |
| ✓ | BbhTrackingThree14CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT650 |
| ✓ | BbhTrackingThree15CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT745 |
| ✓ | BbhTrackingThree16CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT604 |
| ✓ | BbhTrackingThree17CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 197 |
| ✓ | BbhTrackingThree18CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT760 |
| ✓ | BbhTrackingThree19CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT807 |
| ✓ | BbhTrackingThree20CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT1 079 |
| ✓ | BbhTrackingThree21CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT696 |
| ✓ | BbhTrackingThree22CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT618 |
| ✓ | BbhTrackingThree23CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT685 |
| ✓ | BbhTrackingThree24CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 003 |
| ✓ | BbhTrackingThree25CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT779 |
| ✓ | BbhTrackingThree26CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT627 |
| ✓ | BbhTrackingThree27CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT698 |
| ✓ | BbhTrackingThree28CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT966 |
| ✓ | BbhTrackingThree29CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT627 |
| ✓ | BbhTrackingThree30CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 136 |
| ✓ | BbhTrackingThree31CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT803 |
| ✓ | BbhTrackingThree32CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 053 |
| ✓ | BbhTrackingThree33CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 350 |
| ✓ | BbhTrackingThree34CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT838 |
| ✓ | BbhTrackingThree35CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT567 |
| ✓ | BbhTrackingThree36CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT548 |
| ✓ | BbhTrackingThree37CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT714 |
| ✓ | BbhTrackingThree8CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 088 |
| ✓ | BbhTrackingThree9CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 042 |
Reasoning - BBH tracking shuffled objects (3) (set 2)
30 passed |
100.0% |
IN31 998 OUT24 394 | |
| ✓ | BbhTrackingThree38CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT707 |
| ✓ | BbhTrackingThree39CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT732 |
| ✓ | BbhTrackingThree40CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT958 |
| ✓ | BbhTrackingThree41CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT766 |
| ✓ | BbhTrackingThree42CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT735 |
| ✓ | BbhTrackingThree43CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT989 |
| ✓ | BbhTrackingThree44CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT944 |
| ✓ | BbhTrackingThree45CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT671 |
| ✓ | BbhTrackingThree46CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT735 |
| ✓ | BbhTrackingThree47CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT570 |
| ✓ | BbhTrackingThree48CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT750 |
| ✓ | BbhTrackingThree49CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT880 |
| ✓ | BbhTrackingThree50CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT872 |
| ✓ | BbhTrackingThree51CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT637 |
| ✓ | BbhTrackingThree52CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 050 |
| ✓ | BbhTrackingThree53CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 156 |
| ✓ | BbhTrackingThree54CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT936 |
| ✓ | BbhTrackingThree55CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT646 |
| ✓ | BbhTrackingThree56CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT864 |
| ✓ | BbhTrackingThree57CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT737 |
| ✓ | BbhTrackingThree58CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT993 |
| ✓ | BbhTrackingThree59CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT851 |
| ✓ | BbhTrackingThree60CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT898 |
| ✓ | BbhTrackingThree61CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT651 |
| ✓ | BbhTrackingThree62CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT856 |
| ✓ | BbhTrackingThree63CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT810 |
| ✓ | BbhTrackingThree64CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 131 |
| ✓ | BbhTrackingThree65CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT525 |
| ✓ | BbhTrackingThree66CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT625 |
| ✓ | BbhTrackingThree67CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT719 |
Reasoning - BBH tracking shuffled objects (3) (set 3)
30 passed |
100.0% |
IN32 033 OUT24 909 | |
| ✓ | BbhTrackingThree68CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT663 |
| ✓ | BbhTrackingThree69CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT933 |
| ✓ | BbhTrackingThree70CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT682 |
| ✓ | BbhTrackingThree71CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT762 |
| ✓ | BbhTrackingThree72CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT495 |
| ✓ | BbhTrackingThree73CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT706 |
| ✓ | BbhTrackingThree74CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT656 |
| ✓ | BbhTrackingThree75CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT525 |
| ✓ | BbhTrackingThree76CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT726 |
| ✓ | BbhTrackingThree77CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT875 |
| ✓ | BbhTrackingThree78CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT607 |
| ✓ | BbhTrackingThree79CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT997 |
| ✓ | BbhTrackingThree80CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT1 063 |
| ✓ | BbhTrackingThree81CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT920 |
| ✓ | BbhTrackingThree82CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT707 |
| ✓ | BbhTrackingThree83CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT684 |
| ✓ | BbhTrackingThree84CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT612 |
| ✓ | BbhTrackingThree85CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT559 |
| ✓ | BbhTrackingThree86CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT1 298 |
| ✓ | BbhTrackingThree87CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT863 |
| ✓ | BbhTrackingThree88CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT740 |
| ✓ | BbhTrackingThree89CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT983 |
| ✓ | BbhTrackingThree90CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 226 |
| ✓ | BbhTrackingThree91CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT1 054 |
| ✓ | BbhTrackingThree92CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 097 |
| ✓ | BbhTrackingThree93CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT929 |
| ✓ | BbhTrackingThree94CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT725 |
| ✓ | BbhTrackingThree95CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT810 |
| ✓ | BbhTrackingThree96CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT1 124 |
| ✓ | BbhTrackingThree97CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT888 |
Reasoning - BBH tracking shuffled objects (3) (set 4)
29 passed |
100.0% |
IN30 903 OUT25 573 | |
| ✓ | BbhTrackingThree100CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 421 |
| ✓ | BbhTrackingThree101CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT612 |
| ✓ | BbhTrackingThree102CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT725 |
| ✓ | BbhTrackingThree103CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT864 |
| ✓ | BbhTrackingThree104CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 220 |
| ✓ | BbhTrackingThree105CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 612 |
| ✓ | BbhTrackingThree106CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT579 |
| ✓ | BbhTrackingThree107CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT750 |
| ✓ | BbhTrackingThree108CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 064 |
| ✓ | BbhTrackingThree109CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 118 |
| ✓ | BbhTrackingThree110CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 284 |
| ✓ | BbhTrackingThree111CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT744 |
| ✓ | BbhTrackingThree112CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT924 |
| ✓ | BbhTrackingThree113CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT674 |
| ✓ | BbhTrackingThree114CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT739 |
| ✓ | BbhTrackingThree115CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT679 |
| ✓ | BbhTrackingThree116CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT680 |
| ✓ | BbhTrackingThree117CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT711 |
| ✓ | BbhTrackingThree118CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT701 |
| ✓ | BbhTrackingThree119CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT811 |
| ✓ | BbhTrackingThree120CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT783 |
| ✓ | BbhTrackingThree121CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT728 |
| ✓ | BbhTrackingThree122CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT886 |
| ✓ | BbhTrackingThree123CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT656 |
| ✓ | BbhTrackingThree124CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 108 |
| ✓ | BbhTrackingThree125CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 166 |
| ✓ | BbhTrackingThree126CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT694 |
| ✓ | BbhTrackingThree98CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT663 |
| ✓ | BbhTrackingThree99CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT977 |
Reasoning - BBH tracking shuffled objects (3) (set 5)
32 passed |
100.0% |
IN34 179 OUT26 515 | |
| ✓ | BbhTrackThree127CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT562 |
| ✓ | BbhTrackThree128CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT584 |
| ✓ | BbhTrackThree129CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT734 |
| ✓ | BbhTrackThree130CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT822 |
| ✓ | BbhTrackThree131CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT1 577 |
| ✓ | BbhTrackThree132CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT643 |
| ✓ | BbhTrackThree133CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT807 |
| ✓ | BbhTrackThree134CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT620 |
| ✓ | BbhTrackThree135CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT584 |
| ✓ | BbhTrackThree136CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT1 067 |
| ✓ | BbhTrackThree137CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT641 |
| ✓ | BbhTrackThree138CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT590 |
| ✓ | BbhTrackThree139CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT801 |
| ✓ | BbhTrackThree140CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT724 |
| ✓ | BbhTrackThree141CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT885 |
| ✓ | BbhTrackThree142CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT955 |
| ✓ | BbhTrackThree143CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT705 |
| ✓ | BbhTrackThree144CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT780 |
| ✓ | BbhTrackThree145CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT587 |
| ✓ | BbhTrackThree146CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 062 |
| ✓ | BbhTrackThree147CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT974 |
| ✓ | BbhTrackThree148CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT696 |
| ✓ | BbhTrackThree149CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 222 |
| ✓ | BbhTrackThree150CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT929 |
| ✓ | BbhTrackThree151CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT868 |
| ✓ | BbhTrackThree152CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT768 |
| ✓ | BbhTrackThree153CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT922 |
| ✓ | BbhTrackThree154CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT626 |
| ✓ | BbhTrackThree155CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT952 |
| ✓ | BbhTrackThree156CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT959 |
| ✓ | BbhTrackThree157CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT793 |
| ✓ | BbhTrackThree158CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT1 076 |
Reasoning - BBH tracking shuffled objects (3) (set 6)
32 passed |
100.0% |
IN34 087 OUT25 957 | |
| ✓ | BbhTrackThree159CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT689 |
| ✓ | BbhTrackThree160CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 092 |
| ✓ | BbhTrackThree161CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT750 |
| ✓ | BbhTrackThree162CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT723 |
| ✓ | BbhTrackThree163CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT853 |
| ✓ | BbhTrackThree164CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT544 |
| ✓ | BbhTrackThree165CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 037 |
| ✓ | BbhTrackThree166CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT744 |
| ✓ | BbhTrackThree167CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT930 |
| ✓ | BbhTrackThree168CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 248 |
| ✓ | BbhTrackThree169CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT942 |
| ✓ | BbhTrackThree170CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT784 |
| ✓ | BbhTrackThree171CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT854 |
| ✓ | BbhTrackThree172CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT664 |
| ✓ | BbhTrackThree173CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT779 |
| ✓ | BbhTrackThree174CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT565 |
| ✓ | BbhTrackThree175CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 355 |
| ✓ | BbhTrackThree176CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT830 |
| ✓ | BbhTrackThree177CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT774 |
| ✓ | BbhTrackThree178CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT535 |
| ✓ | BbhTrackThree179CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT906 |
| ✓ | BbhTrackThree180CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT733 |
| ✓ | BbhTrackThree181CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT523 |
| ✓ | BbhTrackThree182CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT597 |
| ✓ | BbhTrackThree183CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT810 |
| ✓ | BbhTrackThree184CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT884 |
| ✓ | BbhTrackThree185CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT808 |
| ✓ | BbhTrackThree186CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT563 |
| ✓ | BbhTrackThree187CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT999 |
| ✓ | BbhTrackThree188CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT793 |
| ✓ | BbhTrackThree189CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT871 |
| ✓ | BbhTrackThree190CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT778 |
Reasoning - BBH tracking shuffled objects (3) (set 7)
32 passed |
100.0% |
IN34 106 OUT26 427 | |
| ✓ | BbhTrackThree191CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT741 |
| ✓ | BbhTrackThree192CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT663 |
| ✓ | BbhTrackThree193CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT960 |
| ✓ | BbhTrackThree194CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 395 |
| ✓ | BbhTrackThree195CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT776 |
| ✓ | BbhTrackThree196CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT1 108 |
| ✓ | BbhTrackThree197CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT872 |
| ✓ | BbhTrackThree198CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 323 |
| ✓ | BbhTrackThree199CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT854 |
| ✓ | BbhTrackThree200CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT607 |
| ✓ | BbhTrackThree201CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT693 |
| ✓ | BbhTrackThree202CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT916 |
| ✓ | BbhTrackThree203CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT728 |
| ✓ | BbhTrackThree204CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 231 |
| ✓ | BbhTrackThree205CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT718 |
| ✓ | BbhTrackThree206CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT799 |
| ✓ | BbhTrackThree207CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT841 |
| ✓ | BbhTrackThree208CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT631 |
| ✓ | BbhTrackThree209CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT680 |
| ✓ | BbhTrackThree210CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT829 |
| ✓ | BbhTrackThree211CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT765 |
| ✓ | BbhTrackThree212CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT528 |
| ✓ | BbhTrackThree213CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT709 |
| ✓ | BbhTrackThree214CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT685 |
| ✓ | BbhTrackThree215CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT724 |
| ✓ | BbhTrackThree216CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT814 |
| ✓ | BbhTrackThree217CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT970 |
| ✓ | BbhTrackThree218CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT937 |
| ✓ | BbhTrackThree219CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT738 |
| ✓ | BbhTrackThree220CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT681 |
| ✓ | BbhTrackThree221CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT843 |
| ✓ | BbhTrackThree222CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT668 |
Reasoning - BBH tracking shuffled objects (3) (set 8)
27 passed |
100.0% |
IN28 780 OUT20 580 | |
| ✓ | BbhTrackThree223CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT639 |
| ✓ | BbhTrackThree224CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT636 |
| ✓ | BbhTrackThree225CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT829 |
| ✓ | BbhTrackThree226CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT851 |
| ✓ | BbhTrackThree227CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 073 |
| ✓ | BbhTrackThree228CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT709 |
| ✓ | BbhTrackThree229CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT764 |
| ✓ | BbhTrackThree230CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT696 |
| ✓ | BbhTrackThree231CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT857 |
| ✓ | BbhTrackThree232CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT744 |
| ✓ | BbhTrackThree233CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT738 |
| ✓ | BbhTrackThree234CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT1 102 |
| ✓ | BbhTrackThree235CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT1 058 |
| ✓ | BbhTrackThree236CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT756 |
| ✓ | BbhTrackThree237CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT654 |
| ✓ | BbhTrackThree238CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT771 |
| ✓ | BbhTrackThree239CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT500 |
| ✓ | BbhTrackThree240CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT615 |
| ✓ | BbhTrackThree241CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 115 |
| ✓ | BbhTrackThree242CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT657 |
| ✓ | BbhTrackThree243CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT530 |
| ✓ | BbhTrackThree244CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT822 |
| ✓ | BbhTrackThree245CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT626 |
| ✓ | BbhTrackThree246CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT901 |
| ✓ | BbhTrackThree247CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT629 |
| ✓ | BbhTrackThree248CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT653 |
| ✓ | BbhTrackThree249CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT655 |
Reasoning - BBH tracking shuffled objects (5) (set 1)
33 passed |
100.0% |
IN36 651 OUT44 525 | |
| ✓ | BbhTrackingShuffledFive10CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 003 |
| ✓ | BbhTrackingShuffledFive11CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 054 |
| ✓ | BbhTrackingShuffledFive12CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 412 |
| ✓ | BbhTrackingShuffledFive13CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 249 |
| ✓ | BbhTrackingShuffledFive14CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 283 |
| ✓ | BbhTrackingShuffledFive15CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 134 |
| ✓ | BbhTrackingShuffledFive16CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 552 |
| ✓ | BbhTrackingShuffledFive17CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 358 |
| ✓ | BbhTrackingShuffledFive18CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 300 |
| ✓ | BbhTrackingShuffledFive19CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 116 |
| ✓ | BbhTrackingShuffledFive20CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 060 |
| ✓ | BbhTrackingShuffledFive21CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 506 |
| ✓ | BbhTrackingShuffledFive22CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 298 |
| ✓ | BbhTrackingShuffledFive23CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 001 |
| ✓ | BbhTrackingShuffledFive24CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT3 013 |
| ✓ | BbhTrackingShuffledFive25CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 093 |
| ✓ | BbhTrackingShuffledFive26CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT854 |
| ✓ | BbhTrackingShuffledFive27CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 219 |
| ✓ | BbhTrackingShuffledFive28CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 059 |
| ✓ | BbhTrackingShuffledFive29CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 175 |
| ✓ | BbhTrackingShuffledFive30CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 377 |
| ✓ | BbhTrackingShuffledFive31CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 537 |
| ✓ | BbhTrackingShuffledFive32CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 500 |
| ✓ | BbhTrackingShuffledFive33CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 263 |
| ✓ | BbhTrackingShuffledFive34CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 353 |
| ✓ | BbhTrackingShuffledFive35CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 717 |
| ✓ | BbhTrackingShuffledFive36CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT961 |
| ✓ | BbhTrackingShuffledFive37CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT2 243 |
| ✓ | BbhTrackingShuffledFive38CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 313 |
| ✓ | BbhTrackingShuffledFive39CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 436 |
| ✓ | BbhTrackingShuffledFive40CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT935 |
| ✓ | BbhTrackingShuffledFive8CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 120 |
| ✓ | BbhTrackingShuffledFive9CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 031 |
Reasoning - BBH tracking shuffled objects (5) (set 2)
33 passed |
100.0% |
IN36 557 OUT43 254 | |
| ✓ | BbhTrackingShuffledFive41CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 214 |
| ✓ | BbhTrackingShuffledFive42CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 165 |
| ✓ | BbhTrackingShuffledFive43CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 206 |
| ✓ | BbhTrackingShuffledFive44CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 406 |
| ✓ | BbhTrackingShuffledFive45CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 183 |
| ✓ | BbhTrackingShuffledFive46CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 345 |
| ✓ | BbhTrackingShuffledFive47CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 817 |
| ✓ | BbhTrackingShuffledFive48CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 755 |
| ✓ | BbhTrackingShuffledFive49CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 003 |
| ✓ | BbhTrackingShuffledFive50CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 151 |
| ✓ | BbhTrackingShuffledFive51CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 174 |
| ✓ | BbhTrackingShuffledFive52CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 007 |
| ✓ | BbhTrackingShuffledFive53CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 284 |
| ✓ | BbhTrackingShuffledFive54CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT952 |
| ✓ | BbhTrackingShuffledFive55CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 095 |
| ✓ | BbhTrackingShuffledFive56CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 227 |
| ✓ | BbhTrackingShuffledFive57CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT2 147 |
| ✓ | BbhTrackingShuffledFive58CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 492 |
| ✓ | BbhTrackingShuffledFive59CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 086 |
| ✓ | BbhTrackingShuffledFive60CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 645 |
| ✓ | BbhTrackingShuffledFive61CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 074 |
| ✓ | BbhTrackingShuffledFive62CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 151 |
| ✓ | BbhTrackingShuffledFive63CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 133 |
| ✓ | BbhTrackingShuffledFive64CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 624 |
| ✓ | BbhTrackingShuffledFive65CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 296 |
| ✓ | BbhTrackingShuffledFive66CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 411 |
| ✓ | BbhTrackingShuffledFive67CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 467 |
| ✓ | BbhTrackingShuffledFive68CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 170 |
| ✓ | BbhTrackingShuffledFive69CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 664 |
| ✓ | BbhTrackingShuffledFive70CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 202 |
| ✓ | BbhTrackingShuffledFive71CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 236 |
| ✓ | BbhTrackingShuffledFive72CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 125 |
| ✓ | BbhTrackingShuffledFive73CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 347 |
Reasoning - BBH tracking shuffled objects (5) (set 3)
32 passed |
100.0% |
IN35 526 OUT44 419 | |
| ✓ | BbhTrackingShuffledFive100CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 261 |
| ✓ | BbhTrackingShuffledFive101CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT2 027 |
| ✓ | BbhTrackingShuffledFive102CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT995 |
| ✓ | BbhTrackingShuffledFive103CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 104 |
| ✓ | BbhTrackingShuffledFive104CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 053 |
| ✓ | BbhTrackingShuffledFive105CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 025 |
| ✓ | BbhTrackingShuffledFive74CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT2 018 |
| ✓ | BbhTrackingShuffledFive75CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 141 |
| ✓ | BbhTrackingShuffledFive76CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT994 |
| ✓ | BbhTrackingShuffledFive77CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 077 |
| ✓ | BbhTrackingShuffledFive78CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT2 350 |
| ✓ | BbhTrackingShuffledFive79CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT1 574 |
| ✓ | BbhTrackingShuffledFive80CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 036 |
| ✓ | BbhTrackingShuffledFive81CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 315 |
| ✓ | BbhTrackingShuffledFive82CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 876 |
| ✓ | BbhTrackingShuffledFive83CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 261 |
| ✓ | BbhTrackingShuffledFive84CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT1 062 |
| ✓ | BbhTrackingShuffledFive85CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 342 |
| ✓ | BbhTrackingShuffledFive86CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 269 |
| ✓ | BbhTrackingShuffledFive87CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 004 |
| ✓ | BbhTrackingShuffledFive88CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 935 |
| ✓ | BbhTrackingShuffledFive89CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 090 |
| ✓ | BbhTrackingShuffledFive90CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 313 |
| ✓ | BbhTrackingShuffledFive91CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 209 |
| ✓ | BbhTrackingShuffledFive92CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 633 |
| ✓ | BbhTrackingShuffledFive93CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 726 |
| ✓ | BbhTrackingShuffledFive94CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 563 |
| ✓ | BbhTrackingShuffledFive95CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 447 |
| ✓ | BbhTrackingShuffledFive96CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT948 |
| ✓ | BbhTrackingShuffledFive97CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 132 |
| ✓ | BbhTrackingShuffledFive98CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 468 |
| ✓ | BbhTrackingShuffledFive99CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 171 |
Reasoning - BBH tracking shuffled objects (5) (set 4)
31 passed |
100.0% |
IN34 429 OUT47 423 | |
| ✓ | BbhTrackingShuffledFive106CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 623 |
| ✓ | BbhTrackingShuffledFive107CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 846 |
| ✓ | BbhTrackingShuffledFive108CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 026 |
| ✓ | BbhTrackingShuffledFive109CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 143 |
| ✓ | BbhTrackingShuffledFive110CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 495 |
| ✓ | BbhTrackingShuffledFive111CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 207 |
| ✓ | BbhTrackingShuffledFive112CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT995 |
| ✓ | BbhTrackingShuffledFive113CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 515 |
| ✓ | BbhTrackingShuffledFive114CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 336 |
| ✓ | BbhTrackingShuffledFive115CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT2 052 |
| ✓ | BbhTrackingShuffledFive116CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT1 164 |
| ✓ | BbhTrackingShuffledFive117CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT1 729 |
| ✓ | BbhTrackingShuffledFive118CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 881 |
| ✓ | BbhTrackingShuffledFive119CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 447 |
| ✓ | BbhTrackingShuffledFive120CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 178 |
| ✓ | BbhTrackingShuffledFive121CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 247 |
| ✓ | BbhTrackingShuffledFive122CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 790 |
| ✓ | BbhTrackingShuffledFive123CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 168 |
| ✓ | BbhTrackingShuffledFive124CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 052 |
| ✓ | BbhTrackingShuffledFive125CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 068 |
| ✓ | BbhTrackingShuffledFive126CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT2 031 |
| ✓ | BbhTrackingShuffledFive127CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 810 |
| ✓ | BbhTrackingShuffledFive128CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 597 |
| ✓ | BbhTrackingShuffledFive129CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 382 |
| ✓ | BbhTrackingShuffledFive130CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 680 |
| ✓ | BbhTrackingShuffledFive131CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 020 |
| ✓ | BbhTrackingShuffledFive132CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 103 |
| ✓ | BbhTrackingShuffledFive133CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 622 |
| ✓ | BbhTrackingShuffledFive134CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 559 |
| ✓ | BbhTrackingShuffledFive135CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 385 |
| ✓ | BbhTrackingShuffledFive136CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 272 |
Reasoning - BBH tracking shuffled objects (5) (set 5)
30 passed2 failed |
93.8% |
IN35 499 OUT41 689 | |
| ✓ | BbhTrackFive137CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 224 |
| ✓ | BbhTrackFive138CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 539 |
| ✓ | BbhTrackFive139CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 762 |
| ✓ | BbhTrackFive140CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 256 |
| ✓ | BbhTrackFive141CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 119 |
| ✓ | BbhTrackFive142CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT938 |
| ✓ | BbhTrackFive143CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 440 |
| ✓ | BbhTrackFive144CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 619 |
| ✓ | BbhTrackFive145CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 529 |
| ✓ | BbhTrackFive146CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT1 465 |
| ✓ | BbhTrackFive147CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 047 |
| ✓ | BbhTrackFive148CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 766 |
| ✓ | BbhTrackFive149CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT941 |
| ✓ | BbhTrackFive150CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT989 |
| ✓ | BbhTrackFive151CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 174 |
| ✓ | BbhTrackFive152CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 647 |
| ✓ | BbhTrackFive153CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 416 |
| ✓ | BbhTrackFive154CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 008 |
| ✓ | BbhTrackFive155CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 161 |
| × | BbhTrackFive156CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("")
Context: BbhTrackFive156CN: expected , got [(E)] |
Failed | IN1 103 OUT1 028 |
| × | BbhTrackFive157CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("")
Context: BbhTrackFive157CN: expected , got [(A)] |
Failed | IN1 115 OUT1 157 |
| ✓ | BbhTrackFive158CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 306 |
| ✓ | BbhTrackFive159CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 390 |
| ✓ | BbhTrackFive160CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 543 |
| ✓ | BbhTrackFive161CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 375 |
| ✓ | BbhTrackFive162CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 189 |
| ✓ | BbhTrackFive163CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT976 |
| ✓ | BbhTrackFive164CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 361 |
| ✓ | BbhTrackFive165CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT991 |
| ✓ | BbhTrackFive166CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT1 402 |
| ✓ | BbhTrackFive167CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 279 |
| ✓ | BbhTrackFive168CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 652 |
Reasoning - BBH tracking shuffled objects (5) (set 6)
32 passed |
100.0% |
IN35 543 OUT41 994 | |
| ✓ | BbhTrackFive169CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 540 |
| ✓ | BbhTrackFive170CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 119 |
| ✓ | BbhTrackFive171CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT1 164 |
| ✓ | BbhTrackFive172CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 492 |
| ✓ | BbhTrackFive173CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 136 |
| ✓ | BbhTrackFive174CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 305 |
| ✓ | BbhTrackFive175CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 014 |
| ✓ | BbhTrackFive176CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 194 |
| ✓ | BbhTrackFive177CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 435 |
| ✓ | BbhTrackFive178CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 464 |
| ✓ | BbhTrackFive179CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT1 665 |
| ✓ | BbhTrackFive180CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 144 |
| ✓ | BbhTrackFive181CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 094 |
| ✓ | BbhTrackFive182CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 435 |
| ✓ | BbhTrackFive183CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT2 173 |
| ✓ | BbhTrackFive184CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 629 |
| ✓ | BbhTrackFive185CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 157 |
| ✓ | BbhTrackFive186CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 063 |
| ✓ | BbhTrackFive187CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 095 |
| ✓ | BbhTrackFive188CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 088 |
| ✓ | BbhTrackFive189CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 115 |
| ✓ | BbhTrackFive190CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 099 |
| ✓ | BbhTrackFive191CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 072 |
| ✓ | BbhTrackFive192CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 126 |
| ✓ | BbhTrackFive193CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 849 |
| ✓ | BbhTrackFive194CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT971 |
| ✓ | BbhTrackFive195CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 162 |
| ✓ | BbhTrackFive196CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 348 |
| ✓ | BbhTrackFive197CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 056 |
| ✓ | BbhTrackFive198CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 303 |
| ✓ | BbhTrackFive199CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 362 |
| ✓ | BbhTrackFive200CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 125 |
Reasoning - BBH tracking shuffled objects (5) (set 7)
32 passed |
100.0% |
IN35 504 OUT48 835 | |
| ✓ | BbhTrackFive201CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 059 |
| ✓ | BbhTrackFive202CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 245 |
| ✓ | BbhTrackFive203CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 211 |
| ✓ | BbhTrackFive204CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 474 |
| ✓ | BbhTrackFive205CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 736 |
| ✓ | BbhTrackFive206CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 531 |
| ✓ | BbhTrackFive207CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 193 |
| ✓ | BbhTrackFive208CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 240 |
| ✓ | BbhTrackFive209CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT2 529 |
| ✓ | BbhTrackFive210CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 334 |
| ✓ | BbhTrackFive211CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT2 132 |
| ✓ | BbhTrackFive212CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 013 |
| ✓ | BbhTrackFive213CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 399 |
| ✓ | BbhTrackFive214CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 857 |
| ✓ | BbhTrackFive215CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 272 |
| ✓ | BbhTrackFive216CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT3 117 |
| ✓ | BbhTrackFive217CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 175 |
| ✓ | BbhTrackFive218CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 168 |
| ✓ | BbhTrackFive219CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 137 |
| ✓ | BbhTrackFive220CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 859 |
| ✓ | BbhTrackFive221CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 053 |
| ✓ | BbhTrackFive222CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 172 |
| ✓ | BbhTrackFive223CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT2 520 |
| ✓ | BbhTrackFive224CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 070 |
| ✓ | BbhTrackFive225CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 123 |
| ✓ | BbhTrackFive226CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 124 |
| ✓ | BbhTrackFive227CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 346 |
| ✓ | BbhTrackFive228CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 264 |
| ✓ | BbhTrackFive229CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 331 |
| ✓ | BbhTrackFive230CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 835 |
| ✓ | BbhTrackFive231CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 314 |
| ✓ | BbhTrackFive232CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT2 002 |
Reasoning - BBH tracking shuffled objects (5) (set 8)
32 passed |
100.0% |
IN36 208 OUT52 353 | |
| ✓ | BbhTrackFive233CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 061 |
| ✓ | BbhTrackFive234CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT1 608 |
| ✓ | BbhTrackFive235CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 423 |
| ✓ | BbhTrackFive236CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 078 |
| ✓ | BbhTrackFive237CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 125 |
| ✓ | BbhTrackFive238CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 373 |
| ✓ | BbhTrackFive239CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 222 |
| ✓ | BbhTrackFive240CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 155 |
| ✓ | BbhTrackFive241CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 314 |
| ✓ | BbhTrackFive242CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT2 400 |
| ✓ | BbhTrackFive243CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 573 |
| ✓ | BbhTrackFive244CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT1 644 |
| ✓ | BbhTrackFive245CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 460 |
| ✓ | BbhTrackFive246CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 465 |
| ✓ | BbhTrackFive247CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 560 |
| ✓ | BbhTrackFive248CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 461 |
| ✓ | BbhTrackFive249CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 313 |
| ✓ | BbhTrackSeven10CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 475 |
| ✓ | BbhTrackSeven11CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 637 |
| ✓ | BbhTrackSeven12CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT2 774 |
| ✓ | BbhTrackSeven13CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 738 |
| ✓ | BbhTrackSeven14CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 178 OUT1 786 |
| ✓ | BbhTrackSeven15CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 216 |
| ✓ | BbhTrackSeven16CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 060 |
| ✓ | BbhTrackSeven17CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 693 |
| ✓ | BbhTrackSeven18CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 033 |
| ✓ | BbhTrackSeven19CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 992 |
| ✓ | BbhTrackSeven20CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 947 |
| ✓ | BbhTrackSeven21CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 340 |
| ✓ | BbhTrackSeven22CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 649 |
| ✓ | BbhTrackSeven8CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 309 |
| ✓ | BbhTrackSeven9CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 469 |
Reasoning - BBH tracking shuffled objects (7) (set 1)
30 passed |
100.0% |
IN34 736 OUT59 649 | |
| ✓ | BbhTrackSeven23CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 865 |
| ✓ | BbhTrackSeven24CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 604 |
| ✓ | BbhTrackSeven25CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 865 |
| ✓ | BbhTrackSeven26CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT2 247 |
| ✓ | BbhTrackSeven27CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT2 185 |
| ✓ | BbhTrackSeven28CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 391 |
| ✓ | BbhTrackSeven29CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 218 |
| ✓ | BbhTrackSeven30CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 828 |
| ✓ | BbhTrackSeven31CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 653 |
| ✓ | BbhTrackSeven32CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 568 |
| ✓ | BbhTrackSeven33CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 991 |
| ✓ | BbhTrackSeven34CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 473 |
| ✓ | BbhTrackSeven35CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 137 |
| ✓ | BbhTrackSeven36CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 671 |
| ✓ | BbhTrackSeven37CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 553 |
| ✓ | BbhTrackSeven38CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT3 520 |
| ✓ | BbhTrackSeven39CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 858 |
| ✓ | BbhTrackSeven40CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 476 |
| ✓ | BbhTrackSeven41CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 818 |
| ✓ | BbhTrackSeven42CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 269 |
| ✓ | BbhTrackSeven43CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 806 |
| ✓ | BbhTrackSeven44CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 211 |
| ✓ | BbhTrackSeven45CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT2 044 |
| ✓ | BbhTrackSeven46CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 770 |
| ✓ | BbhTrackSeven47CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT2 871 |
| ✓ | BbhTrackSeven48CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT2 246 |
| ✓ | BbhTrackSeven49CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 676 |
| ✓ | BbhTrackSeven50CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 880 |
| ✓ | BbhTrackSeven51CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT2 331 |
| ✓ | BbhTrackSeven52CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 624 |
Reasoning - BBH tracking shuffled objects (7) (set 2)
32 passed |
100.0% |
IN37 082 OUT60 201 | |
| ✓ | BbhTrackingSeven53COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 691 |
| ✓ | BbhTrackingSeven54COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 176 OUT1 700 |
| ✓ | BbhTrackingSeven55COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 282 |
| ✓ | BbhTrackingSeven56COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 841 |
| ✓ | BbhTrackingSeven57COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 747 |
| ✓ | BbhTrackingSeven58COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 843 |
| ✓ | BbhTrackingSeven59COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT1 561 |
| ✓ | BbhTrackingSeven60COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 312 |
| ✓ | BbhTrackingSeven61COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 369 |
| ✓ | BbhTrackingSeven62COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 988 |
| ✓ | BbhTrackingSeven63COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 652 |
| ✓ | BbhTrackingSeven64COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 424 |
| ✓ | BbhTrackingSeven65COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 183 |
| ✓ | BbhTrackingSeven66COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 351 |
| ✓ | BbhTrackingSeven67COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 697 |
| ✓ | BbhTrackingSeven68COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 341 |
| ✓ | BbhTrackingSeven69COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 444 |
| ✓ | BbhTrackingSeven70COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 391 |
| ✓ | BbhTrackingSeven71COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 819 |
| ✓ | BbhTrackingSeven72COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT2 029 |
| ✓ | BbhTrackingSeven73COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 168 OUT2 311 |
| ✓ | BbhTrackingSeven74COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 520 |
| ✓ | BbhTrackingSeven75COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 745 |
| ✓ | BbhTrackingSeven76COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT2 742 |
| ✓ | BbhTrackingSeven77COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 243 |
| ✓ | BbhTrackingSeven78COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT2 437 |
| ✓ | BbhTrackingSeven79COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 317 |
| ✓ | BbhTrackingSeven80COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 577 |
| ✓ | BbhTrackingSeven81COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 878 |
| ✓ | BbhTrackingSeven82COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 993 |
| ✓ | BbhTrackingSeven83COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT2 977 |
| ✓ | BbhTrackingSeven84COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 796 |
Reasoning - BBH tracking shuffled objects (7) (set 3)
32 passed |
100.0% |
IN37 169 OUT57 028 | |
| ✓ | BbhTrackingSeven100COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 428 |
| ✓ | BbhTrackingSeven101COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 184 OUT1 651 |
| ✓ | BbhTrackingSeven102COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 547 |
| ✓ | BbhTrackingSeven103COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT2 057 |
| ✓ | BbhTrackingSeven104COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 820 |
| ✓ | BbhTrackingSeven105COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 342 |
| ✓ | BbhTrackingSeven106COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 017 |
| ✓ | BbhTrackingSeven107COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 649 |
| ✓ | BbhTrackingSeven108COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT3 299 |
| ✓ | BbhTrackingSeven109COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT1 781 |
| ✓ | BbhTrackingSeven110COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 451 |
| ✓ | BbhTrackingSeven111COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT3 542 |
| ✓ | BbhTrackingSeven112COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 281 |
| ✓ | BbhTrackingSeven113COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 913 |
| ✓ | BbhTrackingSeven114COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 577 |
| ✓ | BbhTrackingSeven115COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 178 OUT1 423 |
| ✓ | BbhTrackingSeven116COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 964 |
| ✓ | BbhTrackingSeven85COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 152 |
| ✓ | BbhTrackingSeven86COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT1 567 |
| ✓ | BbhTrackingSeven87COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 534 |
| ✓ | BbhTrackingSeven88COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 605 |
| ✓ | BbhTrackingSeven89COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT2 025 |
| ✓ | BbhTrackingSeven90COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 789 |
| ✓ | BbhTrackingSeven91COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 702 |
| ✓ | BbhTrackingSeven92COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 741 |
| ✓ | BbhTrackingSeven93COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 441 |
| ✓ | BbhTrackingSeven94COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT2 050 |
| ✓ | BbhTrackingSeven95COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 608 |
| ✓ | BbhTrackingSeven96COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT2 039 |
| ✓ | BbhTrackingSeven97COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 853 |
| ✓ | BbhTrackingSeven98COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 634 |
| ✓ | BbhTrackingSeven99COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 546 |
Reasoning - BBH tracking shuffled objects (7) (set 4)
32 passed |
100.0% |
IN37 165 OUT62 690 | |
| ✓ | BbhTrackingSeven117COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 779 |
| ✓ | BbhTrackingSeven118COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 324 |
| ✓ | BbhTrackingSeven119COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 316 |
| ✓ | BbhTrackingSeven120COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 448 |
| ✓ | BbhTrackingSeven121COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 957 |
| ✓ | BbhTrackingSeven122COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT3 746 |
| ✓ | BbhTrackingSeven123COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 635 |
| ✓ | BbhTrackingSeven124COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 168 OUT1 495 |
| ✓ | BbhTrackingSeven125COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 550 |
| ✓ | BbhTrackingSeven126COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT3 141 |
| ✓ | BbhTrackingSeven127COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 860 |
| ✓ | BbhTrackingSeven128COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 178 OUT1 555 |
| ✓ | BbhTrackingSeven129COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 154 OUT1 428 |
| ✓ | BbhTrackingSeven130COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT2 213 |
| ✓ | BbhTrackingSeven131COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT1 430 |
| ✓ | BbhTrackingSeven132COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 384 |
| ✓ | BbhTrackingSeven133COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 498 |
| ✓ | BbhTrackingSeven134COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 554 |
| ✓ | BbhTrackingSeven135COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 582 |
| ✓ | BbhTrackingSeven136COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 308 |
| ✓ | BbhTrackingSeven137COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 305 |
| ✓ | BbhTrackingSeven138COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT3 563 |
| ✓ | BbhTrackingSeven139COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 182 OUT3 372 |
| ✓ | BbhTrackingSeven140COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT2 158 |
| ✓ | BbhTrackingSeven141COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 083 |
| ✓ | BbhTrackingSeven142COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT1 572 |
| ✓ | BbhTrackingSeven143COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 020 |
| ✓ | BbhTrackingSeven144COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 170 OUT1 785 |
| ✓ | BbhTrackingSeven145COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 228 |
| ✓ | BbhTrackingSeven146COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 019 |
| ✓ | BbhTrackingSeven147COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 983 |
| ✓ | BbhTrackingSeven148COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 399 |
Reasoning - BBH tracking shuffled objects (7) (set 5)
31 passed |
100.0% |
IN35 985 OUT58 473 | |
| ✓ | BbhTrackingSeven149COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 207 |
| ✓ | BbhTrackingSeven150COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 955 |
| ✓ | BbhTrackingSeven151COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT2 079 |
| ✓ | BbhTrackingSeven152COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT3 818 |
| ✓ | BbhTrackingSeven153COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT926 |
| ✓ | BbhTrackingSeven154COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT1 637 |
| ✓ | BbhTrackingSeven155COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 346 |
| ✓ | BbhTrackingSeven156COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT1 602 |
| ✓ | BbhTrackingSeven157COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 954 |
| ✓ | BbhTrackingSeven158COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT3 256 |
| ✓ | BbhTrackingSeven159COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 412 |
| ✓ | BbhTrackingSeven160COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT2 407 |
| ✓ | BbhTrackingSeven161COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 794 |
| ✓ | BbhTrackingSeven162COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 834 |
| ✓ | BbhTrackingSeven163COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 436 |
| ✓ | BbhTrackingSeven164COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT2 435 |
| ✓ | BbhTrackingSeven165COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 400 |
| ✓ | BbhTrackingSeven166COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 807 |
| ✓ | BbhTrackingSeven167COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 360 |
| ✓ | BbhTrackingSeven168COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 279 |
| ✓ | BbhTrackingSeven169COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 598 |
| ✓ | BbhTrackingSeven170COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 546 |
| ✓ | BbhTrackingSeven171COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT2 231 |
| ✓ | BbhTrackingSeven172COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT1 421 |
| ✓ | BbhTrackingSeven173COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 611 |
| ✓ | BbhTrackingSeven174COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 391 |
| ✓ | BbhTrackingSeven175COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 741 |
| ✓ | BbhTrackingSeven176COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 168 OUT1 788 |
| ✓ | BbhTrackingSeven177COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT2 027 |
| ✓ | BbhTrackingSeven178COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 343 |
| ✓ | BbhTrackingSeven179COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 832 |
Reasoning - BBH tracking shuffled objects (7) (set 6)
31 passed |
100.0% |
IN35 993 OUT60 290 | |
| ✓ | BbhTrackingSeven180COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 695 |
| ✓ | BbhTrackingSeven181COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 371 |
| ✓ | BbhTrackingSeven182COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 652 |
| ✓ | BbhTrackingSeven183COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 319 |
| ✓ | BbhTrackingSeven184COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 051 |
| ✓ | BbhTrackingSeven185COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 687 |
| ✓ | BbhTrackingSeven186COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 918 |
| ✓ | BbhTrackingSeven187COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 585 |
| ✓ | BbhTrackingSeven188COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT3 546 |
| ✓ | BbhTrackingSeven189COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 464 |
| ✓ | BbhTrackingSeven190COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT3 674 |
| ✓ | BbhTrackingSeven191COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 821 |
| ✓ | BbhTrackingSeven192COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT2 301 |
| ✓ | BbhTrackingSeven193COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 519 |
| ✓ | BbhTrackingSeven194COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 234 |
| ✓ | BbhTrackingSeven195COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 184 |
| ✓ | BbhTrackingSeven196COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 482 |
| ✓ | BbhTrackingSeven197COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 550 |
| ✓ | BbhTrackingSeven198COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 370 |
| ✓ | BbhTrackingSeven199COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 468 |
| ✓ | BbhTrackingSeven200COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 922 |
| ✓ | BbhTrackingSeven201COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 913 |
| ✓ | BbhTrackingSeven202COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 802 |
| ✓ | BbhTrackingSeven203COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 801 |
| ✓ | BbhTrackingSeven204COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 697 |
| ✓ | BbhTrackingSeven205COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 691 |
| ✓ | BbhTrackingSeven206COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 427 |
| ✓ | BbhTrackingSeven207COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 394 |
| ✓ | BbhTrackingSeven208COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 359 |
| ✓ | BbhTrackingSeven209COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT2 294 |
| ✓ | BbhTrackingSeven210COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT2 099 |
Reasoning - BBH tracking shuffled objects (7) (set 7)
39 passed |
100.0% |
IN45 287 OUT75 897 | |
| ✓ | BbhTrackingSeven211CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 868 |
| ✓ | BbhTrackingSeven212CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 686 |
| ✓ | BbhTrackingSeven213CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT2 040 |
| ✓ | BbhTrackingSeven214CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 262 |
| ✓ | BbhTrackingSeven215CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 387 |
| ✓ | BbhTrackingSeven216CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 762 |
| ✓ | BbhTrackingSeven217CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 168 OUT1 510 |
| ✓ | BbhTrackingSeven218CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 769 |
| ✓ | BbhTrackingSeven219CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 928 |
| ✓ | BbhTrackingSeven220CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 416 |
| ✓ | BbhTrackingSeven221CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 860 |
| ✓ | BbhTrackingSeven222CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 673 |
| ✓ | BbhTrackingSeven223CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT3 281 |
| ✓ | BbhTrackingSeven224CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 907 |
| ✓ | BbhTrackingSeven225CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 170 OUT1 856 |
| ✓ | BbhTrackingSeven226CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 892 |
| ✓ | BbhTrackingSeven227CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT3 951 |
| ✓ | BbhTrackingSeven228CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 668 |
| ✓ | BbhTrackingSeven229CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 628 |
| ✓ | BbhTrackingSeven230CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 301 |
| ✓ | BbhTrackingSeven231CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT1 935 |
| ✓ | BbhTrackingSeven232CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 462 |
| ✓ | BbhTrackingSeven233CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 467 |
| ✓ | BbhTrackingSeven234CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT3 149 |
| ✓ | BbhTrackingSeven235CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 072 |
| ✓ | BbhTrackingSeven236CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 505 |
| ✓ | BbhTrackingSeven237CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 368 |
| ✓ | BbhTrackingSeven238CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 434 |
| ✓ | BbhTrackingSeven239CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT864 |
| ✓ | BbhTrackingSeven240CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 994 |
| ✓ | BbhTrackingSeven241CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT3 337 |
| ✓ | BbhTrackingSeven242CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 738 |
| ✓ | BbhTrackingSeven243CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 248 |
| ✓ | BbhTrackingSeven244CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT2 128 |
| ✓ | BbhTrackingSeven245CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT4 035 |
| ✓ | BbhTrackingSeven246CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 562 |
| ✓ | BbhTrackingSeven247CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT2 034 |
| ✓ | BbhTrackingSeven248CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 390 |
| ✓ | BbhTrackingSeven249CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 530 |
Reasoning - BBH web of lies (set 1)
35 passed |
100.0% |
IN34 922 OUT39 399 | |
| ✓ | BbhWebOfLies10CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 414 |
| ✓ | BbhWebOfLies11CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT975 |
| ✓ | BbhWebOfLies12CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT892 |
| ✓ | BbhWebOfLies13CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 005 |
| ✓ | BbhWebOfLies14CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 517 |
| ✓ | BbhWebOfLies15CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 108 |
| ✓ | BbhWebOfLies16CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 631 |
| ✓ | BbhWebOfLies17CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 109 |
| ✓ | BbhWebOfLies18CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 641 |
| ✓ | BbhWebOfLies19CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 061 |
| ✓ | BbhWebOfLies20CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT761 |
| ✓ | BbhWebOfLies21CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT887 |
| ✓ | BbhWebOfLies22CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 717 |
| ✓ | BbhWebOfLies23CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT943 |
| ✓ | BbhWebOfLies24CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 014 |
| ✓ | BbhWebOfLies25CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 525 |
| ✓ | BbhWebOfLies26CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT895 |
| ✓ | BbhWebOfLies27CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 361 |
| ✓ | BbhWebOfLies28CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 286 |
| ✓ | BbhWebOfLies29CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 951 |
| ✓ | BbhWebOfLies30CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT853 |
| ✓ | BbhWebOfLies31CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 609 |
| ✓ | BbhWebOfLies32CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT941 |
| ✓ | BbhWebOfLies33CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 029 |
| ✓ | BbhWebOfLies34CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT931 |
| ✓ | BbhWebOfLies35CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT770 |
| ✓ | BbhWebOfLies36CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT985 |
| ✓ | BbhWebOfLies37CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT713 |
| ✓ | BbhWebOfLies38CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT955 |
| ✓ | BbhWebOfLies39CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 405 |
| ✓ | BbhWebOfLies40CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT970 |
| ✓ | BbhWebOfLies41CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT812 |
| ✓ | BbhWebOfLies42CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT794 |
| ✓ | BbhWebOfLies8CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 161 |
| ✓ | BbhWebOfLies9CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT778 |
Reasoning - BBH web of lies (set 2)
32 passed |
100.0% |
IN31 928 OUT33 298 | |
| ✓ | BbhWebOfLies43CRtruth-teller/liar chain — Does Audrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT895 |
| ✓ | BbhWebOfLies44CRtruth-teller/liar chain — Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT939 |
| ✓ | BbhWebOfLies45CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT832 |
| ✓ | BbhWebOfLies46CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT981 |
| ✓ | BbhWebOfLies47CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT902 |
| ✓ | BbhWebOfLies48CRtruth-teller/liar chain — Does Maybelle tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 267 |
| ✓ | BbhWebOfLies49CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 375 |
| ✓ | BbhWebOfLies50CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT925 |
| ✓ | BbhWebOfLies51CRtruth-teller/liar chain — Does Sherrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT987 |
| ✓ | BbhWebOfLies52CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT892 |
| ✓ | BbhWebOfLies53CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT892 |
| ✓ | BbhWebOfLies54CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 316 |
| ✓ | BbhWebOfLies55CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT739 |
| ✓ | BbhWebOfLies56CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 364 |
| ✓ | BbhWebOfLies57CRtruth-teller/liar chain — Does Shalonda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT865 |
| ✓ | BbhWebOfLies58CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 193 |
| ✓ | BbhWebOfLies59CRtruth-teller/liar chain — Does Crista tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT741 |
| ✓ | BbhWebOfLies60CRtruth-teller/liar chain — Does Sima tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 351 |
| ✓ | BbhWebOfLies61CRtruth-teller/liar chain — Does Helene tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT908 |
| ✓ | BbhWebOfLies62CRtruth-teller/liar chain — Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT952 |
| ✓ | BbhWebOfLies63CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT924 |
| ✓ | BbhWebOfLies64CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT964 |
| ✓ | BbhWebOfLies65CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 026 |
| ✓ | BbhWebOfLies66CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT758 |
| ✓ | BbhWebOfLies67CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 355 |
| ✓ | BbhWebOfLies68CRtruth-teller/liar chain — Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 225 |
| ✓ | BbhWebOfLies69CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 030 |
| ✓ | BbhWebOfLies70CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT863 |
| ✓ | BbhWebOfLies71CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 608 |
| ✓ | BbhWebOfLies72CRtruth-teller/liar chain — Does Vernell tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 279 |
| ✓ | BbhWebOfLies73CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT995 |
| ✓ | BbhWebOfLies74CRtruth-teller/liar chain — Does Jaymie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT955 |
Reasoning - BBH web of lies (set 3)
31 passed1 failed |
96.9% |
IN31 932 OUT31 575 | |
| ✓ | BbhWebOfLies100CRtruth-teller/liar chain — Does Fidel tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT964 |
| ✓ | BbhWebOfLies101CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT998 |
| ✓ | BbhWebOfLies102CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT982 |
| ✓ | BbhWebOfLies103CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT965 |
| ✓ | BbhWebOfLies104CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT942 |
| ✓ | BbhWebOfLies105CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 009 |
| ✓ | BbhWebOfLies106CRtruth-teller/liar chain — Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 424 |
| ✓ | BbhWebOfLies75CRtruth-teller/liar chain — Does Crista tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT888 |
| ✓ | BbhWebOfLies76CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT716 |
| ✓ | BbhWebOfLies77CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT778 |
| ✓ | BbhWebOfLies78CRtruth-teller/liar chain — Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT851 |
| ✓ | BbhWebOfLies79CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 160 |
| ✓ | BbhWebOfLies80CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 068 |
| ✓ | BbhWebOfLies81CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 188 |
| ✓ | BbhWebOfLies82CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 318 |
| ✓ | BbhWebOfLies83CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 194 |
| ✓ | BbhWebOfLies84CRtruth-teller/liar chain — Does Michael tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT987 |
| ✓ | BbhWebOfLies85CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT674 |
| ✓ | BbhWebOfLies86CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 128 |
| × | BbhWebOfLies87CRtruth-teller/liar chain — Does Leda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("Yes")
Context: BbhWebOfLies87CR: expected Yes, got [true]
Values:
got = "true" |
Failed | IN994 OUT933 |
| ✓ | BbhWebOfLies88CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT698 |
| ✓ | BbhWebOfLies89CRtruth-teller/liar chain — Does Fletcher tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 259 |
| ✓ | BbhWebOfLies90CRtruth-teller/liar chain — Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 150 |
| ✓ | BbhWebOfLies91CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 105 |
| ✓ | BbhWebOfLies92CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT701 |
| ✓ | BbhWebOfLies93CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 149 |
| ✓ | BbhWebOfLies94CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT571 |
| ✓ | BbhWebOfLies95CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT945 |
| ✓ | BbhWebOfLies96CRtruth-teller/liar chain — Does Phoebe tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT937 |
| ✓ | BbhWebOfLies97CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 108 |
| ✓ | BbhWebOfLies98CRtruth-teller/liar chain — Does Leda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT721 |
| ✓ | BbhWebOfLies99CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 064 |
Reasoning - BBH web of lies (set 4)
32 passed |
100.0% |
IN31 952 OUT33 476 | |
| ✓ | BbhWebOfLies107CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 101 |
| ✓ | BbhWebOfLies108CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT997 |
| ✓ | BbhWebOfLies109CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT826 |
| ✓ | BbhWebOfLies110CRtruth-teller/liar chain — Does Kristian tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 408 |
| ✓ | BbhWebOfLies111CRtruth-teller/liar chain — Does Crista tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT946 |
| ✓ | BbhWebOfLies112CRtruth-teller/liar chain — Does Bernita tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 158 |
| ✓ | BbhWebOfLies113CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT865 |
| ✓ | BbhWebOfLies114CRtruth-teller/liar chain — Does Millie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 084 |
| ✓ | BbhWebOfLies115CRtruth-teller/liar chain — Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 201 |
| ✓ | BbhWebOfLies116CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 479 |
| ✓ | BbhWebOfLies117CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT800 |
| ✓ | BbhWebOfLies118CRtruth-teller/liar chain — Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT980 |
| ✓ | BbhWebOfLies119CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT981 |
| ✓ | BbhWebOfLies120CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT951 |
| ✓ | BbhWebOfLies121CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT807 |
| ✓ | BbhWebOfLies122CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT997 |
| ✓ | BbhWebOfLies123CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT907 |
| ✓ | BbhWebOfLies124CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 298 |
| ✓ | BbhWebOfLies125CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT932 |
| ✓ | BbhWebOfLies126CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT947 |
| ✓ | BbhWebOfLies127CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 036 |
| ✓ | BbhWebOfLies128CRtruth-teller/liar chain — Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 180 |
| ✓ | BbhWebOfLies129CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 108 |
| ✓ | BbhWebOfLies130CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 029 |
| ✓ | BbhWebOfLies131CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 059 |
| ✓ | BbhWebOfLies132CRtruth-teller/liar chain — Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 143 |
| ✓ | BbhWebOfLies133CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT904 |
| ✓ | BbhWebOfLies134CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT941 |
| ✓ | BbhWebOfLies135CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT803 |
| ✓ | BbhWebOfLies136CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 319 |
| ✓ | BbhWebOfLies137CRtruth-teller/liar chain — Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 551 |
| ✓ | BbhWebOfLies138CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT738 |
Reasoning - BBH web of lies (set 5)
32 passed |
100.0% |
IN31 946 OUT34 387 | |
| ✓ | BbhWebOfLies139CRtruth-teller/liar chain — Does Fletcher tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT989 |
| ✓ | BbhWebOfLies140CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT867 |
| ✓ | BbhWebOfLies141CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 535 |
| ✓ | BbhWebOfLies142CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 276 |
| ✓ | BbhWebOfLies143CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT851 |
| ✓ | BbhWebOfLies144CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT819 |
| ✓ | BbhWebOfLies145CRtruth-teller/liar chain — Does Fidel tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT904 |
| ✓ | BbhWebOfLies146CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 220 |
| ✓ | BbhWebOfLies147CRtruth-teller/liar chain — Does Millie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT905 |
| ✓ | BbhWebOfLies148CRtruth-teller/liar chain — Does Audrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 066 |
| ✓ | BbhWebOfLies149CRtruth-teller/liar chain — Does Phoebe tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT844 |
| ✓ | BbhWebOfLies150CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT992 |
| ✓ | BbhWebOfLies151CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT879 |
| ✓ | BbhWebOfLies152CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 019 |
| ✓ | BbhWebOfLies153CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 493 |
| ✓ | BbhWebOfLies154CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 717 |
| ✓ | BbhWebOfLies155CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT764 |
| ✓ | BbhWebOfLies156CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 055 |
| ✓ | BbhWebOfLies157CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT715 |
| ✓ | BbhWebOfLies158CRtruth-teller/liar chain — Does Shalonda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 526 |
| ✓ | BbhWebOfLies159CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 344 |
| ✓ | BbhWebOfLies160CRtruth-teller/liar chain — Does Ryan tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 006 |
| ✓ | BbhWebOfLies161CRtruth-teller/liar chain — Does Willian tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 234 |
| ✓ | BbhWebOfLies162CRtruth-teller/liar chain — Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 375 |
| ✓ | BbhWebOfLies163CRtruth-teller/liar chain — Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT993 |
| ✓ | BbhWebOfLies164CRtruth-teller/liar chain — Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 194 |
| ✓ | BbhWebOfLies165CRtruth-teller/liar chain — Does Raymond tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT899 |
| ✓ | BbhWebOfLies166CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT854 |
| ✓ | BbhWebOfLies167CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 140 |
| ✓ | BbhWebOfLies168CRtruth-teller/liar chain — Does Leda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT928 |
| ✓ | BbhWebOfLies169CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 035 |
| ✓ | BbhWebOfLies170CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT949 |
Reasoning - BBH web of lies (set 6)
30 passed |
100.0% |
IN29 926 OUT35 145 | |
| ✓ | BbhWebOfLies171CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 136 |
| ✓ | BbhWebOfLies172CRtruth-teller/liar chain — Does Bernita tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 054 |
| ✓ | BbhWebOfLies173CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT973 |
| ✓ | BbhWebOfLies174CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 000 |
| ✓ | BbhWebOfLies175CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT724 |
| ✓ | BbhWebOfLies176CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT903 |
| ✓ | BbhWebOfLies177CRtruth-teller/liar chain — Does Phoebe tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 065 |
| ✓ | BbhWebOfLies178CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT852 |
| ✓ | BbhWebOfLies179CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 602 |
| ✓ | BbhWebOfLies180CRtruth-teller/liar chain — Does Sherrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT773 |
| ✓ | BbhWebOfLies181CRtruth-teller/liar chain — Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT778 |
| ✓ | BbhWebOfLies182CRtruth-teller/liar chain — Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT769 |
| ✓ | BbhWebOfLies183CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 428 |
| ✓ | BbhWebOfLies184CRtruth-teller/liar chain — Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT915 |
| ✓ | BbhWebOfLies185CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 474 |
| ✓ | BbhWebOfLies186CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 368 |
| ✓ | BbhWebOfLies187CRtruth-teller/liar chain — Does Fletcher tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT998 |
| ✓ | BbhWebOfLies188CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT889 |
| ✓ | BbhWebOfLies189CRtruth-teller/liar chain — Does Helene tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 288 |
| ✓ | BbhWebOfLies190CRtruth-teller/liar chain — Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT2 766 |
| ✓ | BbhWebOfLies191CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT918 |
| ✓ | BbhWebOfLies192CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT747 |
| ✓ | BbhWebOfLies193CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT848 |
| ✓ | BbhWebOfLies194CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 215 |
| ✓ | BbhWebOfLies195CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 849 |
| ✓ | BbhWebOfLies196CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 036 |
| ✓ | BbhWebOfLies197CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 269 |
| ✓ | BbhWebOfLies198CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT880 |
| ✓ | BbhWebOfLies199CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 028 |
| ✓ | BbhWebOfLies200CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 600 |
Reasoning - BBH web of lies (set 7)
32 passed |
100.0% |
IN31 938 OUT36 844 | |
| ✓ | BbhWebOfLies201CSJamey tells the truth. Michael says Jamey lies. Sherrie says Michael lies. Bernita says Sherrie lies. Amberly says Bernita tells the truth. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT861 |
| ✓ | BbhWebOfLies202CSShaunda tells the truth. Jim says Shaunda lies. Alexis says Jim lies. Phoebe says Alexis tells the truth. Sherrie says Phoebe tells the truth. Does Sherrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT838 |
| ✓ | BbhWebOfLies203CSMillicent lies. Ka says Millicent lies. Sima says Ka tells the truth. Dallas says Sima tells the truth. Jamey says Dallas lies. Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 289 |
| ✓ | BbhWebOfLies204CSChristie tells the truth. Fidel says Christie lies. Jerry says Fidel tells the truth. Maybelle says Jerry lies. Delfina says Maybelle tells the truth. Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT794 |
| ✓ | BbhWebOfLies205CSSima lies. Maybelle says Sima tells the truth. Audrie says Maybelle lies. Leda says Audrie tells the truth. Shaunda says Leda lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 397 |
| ✓ | BbhWebOfLies206CSVernell lies. Conception says Vernell lies. Kandi says Conception lies. Michael says Kandi tells the truth. Osvaldo says Michael tells the truth. Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 283 |
| ✓ | BbhWebOfLies207CSShaunda tells the truth. Delbert says Shaunda tells the truth. Bernita says Delbert lies. Yoland says Bernita tells the truth. Christie says Yoland tells the truth. Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 056 |
| ✓ | BbhWebOfLies208CSInga tells the truth. Shalonda says Inga tells the truth. Phoebe says Shalonda tells the truth. Crista says Phoebe lies. Alejandro says Crista lies. Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT916 |
| ✓ | BbhWebOfLies209CSFidel lies. Millicent says Fidel tells the truth. Crista says Millicent lies. Leda says Crista lies. Tamika says Leda tells the truth. Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 258 |
| ✓ | BbhWebOfLies210CSAndree tells the truth. Christie says Andree tells the truth. Fidel says Christie lies. Antwan says Fidel tells the truth. Alejandro says Antwan lies. Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT926 |
| ✓ | BbhWebOfLies211CSFletcher tells the truth. Willian says Fletcher tells the truth. Jamey says Willian lies. Leda says Jamey lies. Jaymie says Leda tells the truth. Does Jaymie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 135 |
| ✓ | BbhWebOfLies212CSTamika lies. Jerry says Tamika tells the truth. Delbert says Jerry tells the truth. Leda says Delbert tells the truth. Willian says Leda lies. Does Willian tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 311 |
| ✓ | BbhWebOfLies213CSTeressa lies. Sal says Teressa tells the truth. Delbert says Sal tells the truth. Tamika says Delbert tells the truth. Gwenn says Tamika tells the truth. Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 260 |
| ✓ | BbhWebOfLies214CSJim lies. Delfina says Jim tells the truth. Shaunda says Delfina lies. Fidel says Shaunda lies. Sal says Fidel tells the truth. Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT959 |
| ✓ | BbhWebOfLies215CSMillie lies. Michael says Millie tells the truth. Vina says Michael lies. Conception says Vina tells the truth. Sima says Conception tells the truth. Does Sima tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 094 |
| ✓ | BbhWebOfLies216CSKandi lies. Millie says Kandi tells the truth. Lorine says Millie tells the truth. Michaela says Lorine tells the truth. Amberly says Michaela lies. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 012 |
| ✓ | BbhWebOfLies217CSMichael lies. Jerry says Michael tells the truth. Delbert says Jerry tells the truth. Sima says Delbert lies. Alexis says Sima lies. Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT2 474 |
| ✓ | BbhWebOfLies218CSSal lies. Kristian says Sal lies. Helene says Kristian lies. Inga says Helene tells the truth. Michael says Inga tells the truth. Does Michael tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT670 |
| ✓ | BbhWebOfLies219CSMaybelle lies. Crista says Maybelle tells the truth. Bernita says Crista lies. Kandi says Bernita tells the truth. Delbert says Kandi tells the truth. Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 192 |
| ✓ | BbhWebOfLies220CSKa lies. Rashida says Ka tells the truth. Sherrie says Rashida tells the truth. Tamika says Sherrie tells the truth. Elanor says Tamika tells the truth. Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 195 |
| ✓ | BbhWebOfLies221CSAntwan tells the truth. Audrie says Antwan tells the truth. Inga says Audrie lies. Shenna says Inga tells the truth. Christie says Shenna lies. Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 212 |
| ✓ | BbhWebOfLies222CSTeressa lies. Michael says Teressa tells the truth. Fidel says Michael tells the truth. Leda says Fidel lies. Lorine says Leda lies. Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 531 |
| ✓ | BbhWebOfLies223CSAntwan tells the truth. Elanor says Antwan tells the truth. Bernita says Elanor tells the truth. Osvaldo says Bernita lies. Christie says Osvaldo tells the truth. Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 420 |
| ✓ | BbhWebOfLies224CSInga tells the truth. Elanor says Inga lies. Millicent says Elanor tells the truth. Willian says Millicent tells the truth. Alejandro says Willian tells the truth. Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 209 |
| ✓ | BbhWebOfLies225CSSherrie lies. Elanor says Sherrie tells the truth. Michael says Elanor tells the truth. Sima says Michael tells the truth. Shaunda says Sima lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 074 |
| ✓ | BbhWebOfLies226CSTamika lies. Conception says Tamika tells the truth. Kristian says Conception lies. Alejandro says Kristian tells the truth. Shaunda says Alejandro lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 222 |
| ✓ | BbhWebOfLies227CSHelene lies. Delfina says Helene lies. Christie says Delfina tells the truth. Jerry says Christie lies. Tamika says Jerry tells the truth. Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 153 |
| ✓ | BbhWebOfLies228CSConception lies. Shenna says Conception tells the truth. Maybelle says Shenna tells the truth. Michael says Maybelle lies. Amberly says Michael lies. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 096 |
| ✓ | BbhWebOfLies229CSKristian tells the truth. Jamey says Kristian tells the truth. Millicent says Jamey lies. Fidel says Millicent lies. Tamika says Fidel lies. Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 214 |
| ✓ | BbhWebOfLies230CSJamey lies. Sal says Jamey tells the truth. Teressa says Sal lies. Maybelle says Teressa tells the truth. Ka says Maybelle lies. Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 132 |
| ✓ | BbhWebOfLies231CSLorine lies. Amberly says Lorine lies. Fletcher says Amberly lies. Millie says Fletcher tells the truth. Antwan says Millie lies. Does Antwan tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT856 |
| ✓ | BbhWebOfLies232CSAlexis tells the truth. Elanor says Alexis tells the truth. Sherrie says Elanor lies. Audrie says Sherrie tells the truth. Inga says Audrie tells the truth. Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT805 |
Reasoning - BBH web of lies (set 8)
30 passed2 failed |
93.8% |
IN31 665 OUT32 684 | |
| ✓ | BbhWebOfLies233CSFidel tells the truth. Teressa says Fidel tells the truth. Willian says Teressa tells the truth. Conception says Willian tells the truth. Bernita says Conception lies. Does Bernita tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 123 |
| ✓ | BbhWebOfLies234CSJerry tells the truth. Lorine says Jerry lies. Shenna says Lorine lies. Gwenn says Shenna lies. Shaunda says Gwenn lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT589 |
| ✓ | BbhWebOfLies235CSInga lies. Audrie says Inga lies. Rashida says Audrie lies. Willian says Rashida tells the truth. Amberly says Willian lies. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT2 085 |
| ✓ | BbhWebOfLies236CSKa tells the truth. Inga says Ka lies. Bernita says Inga lies. Osvaldo says Bernita tells the truth. Michaela says Osvaldo lies. Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 108 |
| ✓ | BbhWebOfLies237CSJaymie lies. Jamey says Jaymie tells the truth. Sal says Jamey lies. Amberly says Sal lies. Jerry says Amberly tells the truth. Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 059 |
| ✓ | BbhWebOfLies238CSMillie lies. Andree says Millie lies. Michaela says Andree lies. Leda says Michaela tells the truth. Shenna says Leda lies. Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT899 |
| ✓ | BbhWebOfLies239CSShaunda tells the truth. Maybelle says Shaunda tells the truth. Lorine says Maybelle lies. Millie says Lorine tells the truth. Vina says Millie tells the truth. Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT969 |
| ✓ | BbhWebOfLies240CSFletcher lies. Tamika says Fletcher tells the truth. Ka says Tamika lies. Sima says Ka tells the truth. Ryan says Sima tells the truth. Does Ryan tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT926 |
| ✓ | BbhWebOfLies241CSShaunda lies. Andree says Shaunda tells the truth. Jaymie says Andree lies. Inga says Jaymie lies. Millicent says Inga tells the truth. Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 063 |
| ✓ | BbhWebOfLies242CSRashida lies. Fletcher says Rashida lies. Antwan says Fletcher lies. Willian says Antwan tells the truth. Vina says Willian tells the truth. Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 608 |
| ✓ | BbhWebOfLies243CSJerry lies. Kandi says Jerry lies. Fletcher says Kandi lies. Ryan says Fletcher lies. Delfina says Ryan lies. Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 127 |
| ✓ | BbhWebOfLies244CSShaunda tells the truth. Phoebe says Shaunda tells the truth. Ryan says Phoebe tells the truth. Christie says Ryan tells the truth. Vina says Christie tells the truth. Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 008 |
| ✓ | BbhWebOfLies245CSAlexis tells the truth. Amberly says Alexis tells the truth. Andree says Amberly tells the truth. Vina says Andree tells the truth. Helene says Vina tells the truth. Does Helene tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 195 |
| ✓ | BbhWebOfLies246CSGwenn lies. Maybelle says Gwenn tells the truth. Raymond says Maybelle tells the truth. Jamey says Raymond tells the truth. Andree says Jamey lies. Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 173 |
| ✓ | BbhWebOfLies247CSKandi tells the truth. Ryan says Kandi lies. Michaela says Ryan tells the truth. Sima says Michaela lies. Andree says Sima tells the truth. Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 126 |
| ✓ | BbhWebOfLies248CSFletcher tells the truth. Alexis says Fletcher lies. Jerry says Alexis lies. Elanor says Jerry tells the truth. Jim says Elanor lies. Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT857 |
| ✓ | BbhWebOfLies249CSSherrie tells the truth. Ryan says Sherrie lies. Bernita says Ryan tells the truth. Tamika says Bernita lies. Jerry says Tamika lies. Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT716 |
| ✓ | BbhWordSorting10CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT834 |
| × | BbhWordSorting11CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"allotted fate figural gorky grapple hydroxyl knives neapolitan nerve plainfield rampage saxon scottish scrumptious seventeen sidereal siena stooge thermal yakima")
Context: BbhWordSorting11CS: expected allotted fate figural gorky grapple hydroxyl knives neapolitan nerve plainfield rampage saxon scottish scrumptious seventeen sidereal siena stooge thermal yakima, got [allotted fate figural grapple gorky hydroxyl knives neapolitan nerve plainfield rampage saxon scottish scrumptious seventeen siena sidereal stooge thermal yakima] |
Failed | IN1 000 OUT1 970 |
| ✓ | BbhWordSorting12CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 155 |
| ✓ | BbhWordSorting13CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT958 |
| ✓ | BbhWordSorting14CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT727 |
| ✓ | BbhWordSorting15CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 256 |
| × | BbhWordSorting16CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"advent anger convoy deliver filly gneiss grocer hessian hotbox landau marlborough ninebark plat platelet pyrotechnic siemens stapleton transitive treadle uncle")
Context: BbhWordSorting16CS: expected advent anger convoy deliver filly gneiss grocer hessian hotbox landau marlborough ninebark plat platelet pyrotechnic siemens stapleton transitive treadle uncle, got [advent anger convoy deliver filly gneiss grocer hessian hotbox landau marlborough ninebark plat platelet pyrotechnic siemens stapleton treadle transitive uncle] |
Failed | IN1 001 OUT1 219 |
| ✓ | BbhWordSorting17CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT912 |
| ✓ | BbhWordSorting18CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT1 467 |
| ✓ | BbhWordSorting19CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT553 |
| ✓ | BbhWordSorting20CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT520 |
| ✓ | BbhWordSorting21CSalphabetically sort 7 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT605 |
| ✓ | BbhWordSorting22CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT939 |
| ✓ | BbhWordSorting8CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT567 |
| ✓ | BbhWordSorting9CSalphabetically sort 6 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT371 |
Reasoning - BBH word sorting (set 1)
31 passed1 failed |
96.9% |
IN31 387 OUT33 904 | |
| ✓ | BbhWordSorting23CSalphabetically sort 7 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT841 |
| ✓ | BbhWordSorting24CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT2 018 |
| ✓ | BbhWordSorting25CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT2 177 |
| ✓ | BbhWordSorting26CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 081 |
| ✓ | BbhWordSorting27CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT429 |
| ✓ | BbhWordSorting28CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT796 |
| ✓ | BbhWordSorting29CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 614 |
| ✓ | BbhWordSorting30CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT1 366 |
| ✓ | BbhWordSorting31CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 361 |
| ✓ | BbhWordSorting32CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN965 OUT369 |
| ✓ | BbhWordSorting33CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT1 192 |
| ✓ | BbhWordSorting34CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT600 |
| × | BbhWordSorting35CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"belize bolshevism cost dance deadline dietetic formulae foster hesitant huddle judson mantle odessa palace progeny proust rackety resplendent thirdhand warmth")
Context: BbhWordSorting35CS: expected belize bolshevism cost dance deadline dietetic formulae foster hesitant huddle judson mantle odessa palace progeny proust rackety resplendent thirdhand warmth, got [belize bolshevism cost dance deadline dietetic foster formulae hesitant huddle judson mantle odessa palace progeny proust rackety resplendent thirdhand warmth] |
Failed | IN996 OUT1 867 |
| ✓ | BbhWordSorting36CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT964 |
| ✓ | BbhWordSorting37CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 961 |
| ✓ | BbhWordSorting38CSalphabetically sort 2 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT505 |
| ✓ | BbhWordSorting39CSalphabetically sort 2 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN964 OUT419 |
| ✓ | BbhWordSorting40CSalphabetically sort 14 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 290 |
| ✓ | BbhWordSorting41CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 506 |
| ✓ | BbhWordSorting42CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT467 |
| ✓ | BbhWordSorting43CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT647 |
| ✓ | BbhWordSorting44CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT2 059 |
| ✓ | BbhWordSorting45CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT924 |
| ✓ | BbhWordSorting46CSalphabetically sort 6 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT944 |
| ✓ | BbhWordSorting47CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT625 |
| ✓ | BbhWordSorting48CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT907 |
| ✓ | BbhWordSorting49CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 442 |
| ✓ | BbhWordSorting50CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT504 |
| ✓ | BbhWordSorting51CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT673 |
| ✓ | BbhWordSorting52CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT596 |
| ✓ | BbhWordSorting53CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT511 |
| ✓ | BbhWordSorting54CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 249 |
Reasoning - BBH word sorting (set 2)
25 passed7 failed |
78.1% |
IN31 603 OUT44 276 | |
| × | BbhWordSorting55CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"brake bunny chevalier choose cubby drone exotica good indomitable lineage mental nick pique rajah sheaf steelmake stitch thymine wigging")
Context: BbhWordSorting55CS: expected brake bunny chevalier choose cubby drone exotica good indomitable lineage mental nick pique rajah sheaf steelmake stitch thymine wigging, got [brake, bunny, chevalier, choose, cubby, drone, exotica, good, indomitable, lineage, mental, nick, pique, rajah, sheaf, stitch, steelmak e?, thymine, wigging] |
Failed | IN994 OUT4 896 |
| ✓ | BbhWordSorting56CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT704 |
| ✓ | BbhWordSorting57CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 010 OUT1 238 |
| × | BbhWordSorting58CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"auxin awash bateau cubit eutectic gown gullible inane jurisprudential mistletoe nepenthe ow pirouette pussycat schwartz scottsdale shockley travelogue upbring")
Context: BbhWordSorting58CS: expected auxin awash bateau cubit eutectic gown gullible inane jurisprudential mistletoe nepenthe ow pirouette pussycat schwartz scottsdale shockley travelogue upbring, got [awash auxin bateau cubit eutectic gown gullible inane jurisprudential mistletoe nepenthe ow pirouette pussycat schwartz scottsdale shockley travelogue upbring] |
Failed | IN1 005 OUT1 781 |
| ✓ | BbhWordSorting59CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT1 042 |
| ✓ | BbhWordSorting60CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT719 |
| ✓ | BbhWordSorting61CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 338 |
| ✓ | BbhWordSorting62CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 001 OUT1 415 |
| ✓ | BbhWordSorting63CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 642 |
| ✓ | BbhWordSorting64CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT908 |
| ✓ | BbhWordSorting65CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 205 |
| ✓ | BbhWordSorting66CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 452 |
| × | BbhWordSorting67CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"allele anthropocentric badinage banish bartok brunswick dale dar desolater dun fraternity goat martinson monomer morphemic pegging starkey underclassmen whoop yourselves")
Context: BbhWordSorting67CS: expected allele anthropocentric badinage banish bartok brunswick dale dar desolater dun fraternity goat martinson monomer morphemic pegging starkey underclassmen whoop yourselves, got [allele anthropocentric badinage banish bartok brunswick dale dar desolater dun fraternity goat martinson morphemic monomer pegging starkey underclassmen whoop yourselves] |
Failed | IN999 OUT1 832 |
| ✓ | BbhWordSorting68CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT628 |
| × | BbhWordSorting69CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"avalanche befriend berniece bong bremsstrahlung dactylic flick gilbertson goff hereafter hoe housekeep hurry lanka metazoan posterior showroom")
Context: BbhWordSorting69CS: expected avalanche befriend berniece bong bremsstrahlung dactylic flick gilbertson goff hereafter hoe housekeep hurry lanka metazoan posterior showroom, got [avalanche befriend berniece bremsstrahlung bong dactylic flick gilbertson goff hereafter hoe housekeep hurry lanka metazoan posterior showroom] |
Failed | IN994 OUT1 243 |
| ✓ | BbhWordSorting70CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 683 |
| ✓ | BbhWordSorting71CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 015 |
| ✓ | BbhWordSorting72CSalphabetically sort 8 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 253 |
| ✓ | BbhWordSorting73CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT1 407 |
| × | BbhWordSorting74CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"allot chauncey clergymen coachmen coddington companion embark fatten gazpacho granular hobble murk muslim niggle pristine pvc singlet threefold too yeats")
Context: BbhWordSorting74CS: expected allot chauncey clergymen coachmen coddington companion embark fatten gazpacho granular hobble murk muslim niggle pristine pvc singlet threefold too yeats, got [allot chauncey coachmen coddington clergymen companion embark fatten gazpacho granular hobble murk muslim niggle pristine pvc singlet threefold too yeats] |
Failed | IN1 001 OUT2 131 |
| ✓ | BbhWordSorting75CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT566 |
| × | BbhWordSorting76CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"astigmat boyish coriolanus creak cutlet easternmost godson heaven highwaymen leather muscular musky paula scavenge synaptic zinc")
Context: BbhWordSorting76CS: expected astigmat boyish coriolanus creak cutlet easternmost godson heaven highwaymen leather muscular musky paula scavenge synaptic zinc, got [astigmat boyish coriolanus cutlet creak easternmost godson heaven highwaymen leather muscular musky paula scavenge synaptic zinc] |
Failed | IN991 OUT1 392 |
| ✓ | BbhWordSorting77CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT486 |
| ✓ | BbhWordSorting78CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 230 |
| ✓ | BbhWordSorting79CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 943 |
| ✓ | BbhWordSorting80CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 006 OUT1 273 |
| ✓ | BbhWordSorting81CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT2 024 |
| ✓ | BbhWordSorting82CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT655 |
| ✓ | BbhWordSorting83CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 415 |
| ✓ | BbhWordSorting84CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT541 |
| ✓ | BbhWordSorting85CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 460 |
| × | BbhWordSorting86CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"betony boar bootleg bronzy centaur charge clemens collet contemporaneous contravariant cordial dorado handicraft macmahon mesh monterey possession regina underclassman")
Context: BbhWordSorting86CS: expected betony boar bootleg bronzy centaur charge clemens collet contemporaneous contravariant cordial dorado handicraft macmahon mesh monterey possession regina underclassman, got [betony, boar, bootleg, bronzy, charge, centaur, clemens, collet, contemporaneous, contravariant, cordial, dorado, handicraft, macmahon, mesh, monterey, possession, regina, underclassman] |
Failed | IN1 000 OUT1 759 |
Reasoning - BBH word sorting (set 3)
24 passed6 failed |
80.0% |
IN29 591 OUT35 132 | |
| ✓ | BbhWordSorting100CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT592 |
| × | BbhWordSorting101CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"administer aeneid coachman decadent delhi dey gradate grim jacky littleneck phosphorescent pristine shrunk sinh systemwide tasting thrown torpedo verdict")
Context: BbhWordSorting101CS: expected administer aeneid coachman decadent delhi dey gradate grim jacky littleneck phosphorescent pristine shrunk sinh systemwide tasting thrown torpedo verdict, got [administer aeneid coachman decadent delhi dey gradate grim jacky littleneck phosphorescent pristine shrunk sinh systemwide tasting torpedo thrown verdict] |
Failed | IN993 OUT1 107 |
| ✓ | BbhWordSorting102CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 004 OUT1 847 |
| ✓ | BbhWordSorting103CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT772 |
| ✓ | BbhWordSorting104CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 151 |
| ✓ | BbhWordSorting105CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT910 |
| × | BbhWordSorting106CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"abutted agamemnon aquatic capacity casualty essex guinea hitachi hondo islamic loosen loquacious niece planet roadway solstice steed suspicion tibet")
Context: BbhWordSorting106CS: expected abutted agamemnon aquatic capacity casualty essex guinea hitachi hondo islamic loosen loquacious niece planet roadway solstice steed suspicion tibet, got [agamemnon aquatic abutted casualty capacity essex guinea hitachi hondo islamic loosen loquacious niece planet roadway solstice steed suspicion tibet] |
Failed | IN996 OUT1 305 |
| ✓ | BbhWordSorting107CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT879 |
| ✓ | BbhWordSorting108CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT1 065 |
| × | BbhWordSorting109CSalphabetically sort 8 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"blutwurst buckaroo closeup intelligent laguerre thesaurus vertebral wily")
Context: BbhWordSorting109CS: expected blutwurst buckaroo closeup intelligent laguerre thesaurus vertebral wily, got [buckaroo blutwurst closeup intelligent laguerre thesaurus vertebral wily] |
Failed | IN981 OUT859 |
| ✓ | BbhWordSorting110CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT721 |
| ✓ | BbhWordSorting111CSalphabetically sort 14 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT1 035 |
| ✓ | BbhWordSorting112CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT1 721 |
| ✓ | BbhWordSorting113CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 638 |
| ✓ | BbhWordSorting114CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT1 390 |
| ✓ | BbhWordSorting115CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT807 |
| ✓ | BbhWordSorting116CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 652 |
| ✓ | BbhWordSorting87CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 710 |
| ✓ | BbhWordSorting88CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT469 |
| × | BbhWordSorting89CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"ambient appropriable arroyo billion breccia coupon eardrum faze fivefold intimidate martinson o'connor perplex secretary social surtout terrestrial voltmeter")
Context: BbhWordSorting89CS: expected ambient appropriable arroyo billion breccia coupon eardrum faze fivefold intimidate martinson o'connor perplex secretary social surtout terrestrial voltmeter, got [ambient arroyo appropriable billion breccia coupon eardrum faze fivefold intimidate martinson o'connor perplex secretary social surtout terrestrial voltmeter] |
Failed | IN995 OUT2 014 |
| × | BbhWordSorting90CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"assure bully butterball bye contend cornet deaf dinosaur frontage gunky indeterminable lustrous ostentatious paradigmatic rhyme sanderson sashimi smokestack taint")
Context: BbhWordSorting90CS: expected assure bully butterball bye contend cornet deaf dinosaur frontage gunky indeterminable lustrous ostentatious paradigmatic rhyme sanderson sashimi smokestack taint, got [assure bully butterball bye contend cornet deaf dinosaur frontage gunky indeterminable lustrous ostentatious paradigmatic rhyme sashimi sanderson smokestack taint] |
Failed | IN997 OUT1 543 |
| ✓ | BbhWordSorting91CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT1 138 |
| ✓ | BbhWordSorting92CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT525 |
| ✓ | BbhWordSorting93CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 928 |
| ✓ | BbhWordSorting94CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 040 |
| ✓ | BbhWordSorting95CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 454 |
| ✓ | BbhWordSorting96CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 330 |
| ✓ | BbhWordSorting97CSalphabetically sort 14 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 489 |
| ✓ | BbhWordSorting98CSalphabetically sort 2 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN964 OUT470 |
| × | BbhWordSorting99CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"lise miaow snipe")
Context: BbhWordSorting99CS: expected lise miaow snipe, got [miaow lise snipe] |
Failed | IN968 OUT571 |
Reasoning - BBH word sorting (set 4)
27 passed |
100.0% |
IN26 516 OUT28 286 | |
| ✓ | BbhWordSorting117CTSort the following words alphabetically: List: marshmallow doge alpenstock ocean accept angus drool jutish resistive chromium vociferous castigate prize octennial stonewort concision elizabethan1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT1 289 |
| ✓ | BbhWordSorting118CTSort the following words alphabetically: List: jocund flagellate bodyguard flotation commensal involve miff ineradicable postprocess1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 015 |
| ✓ | BbhWordSorting119CTSort the following words alphabetically: List: gloriana milord rendezvous schnabel allegoric loge lollipop salle collate prolix mast euphony1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 105 |
| ✓ | BbhWordSorting120CTSort the following words alphabetically: List: pea yonkers darkle quadrant moldboard instant olsen hookup erudite lip1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT801 |
| ✓ | BbhWordSorting121CTSort the following words alphabetically: List: regret uptake satisfy behold fumarole motif forth alcohol escutcheon pease hackberry walkie1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 194 |
| ✓ | BbhWordSorting122CTSort the following words alphabetically: List: charcuterie muddle diatom impenetrable footfall crucifix greenberg synchronous spoken1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 202 |
| ✓ | BbhWordSorting123CTSort the following words alphabetically: List: sibyl calligraph goat inverness form threadbare1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT556 |
| ✓ | BbhWordSorting124CTSort the following words alphabetically: List: cashew apprehension ensemble1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT571 |
| ✓ | BbhWordSorting125CTSort the following words alphabetically: List: seashore haughty1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT484 |
| ✓ | BbhWordSorting126CTSort the following words alphabetically: List: postcondition protoplasmic musicology helical uptrend vasoconstriction diacritic beefsteak beware birthplace bicycle junctor state obstinate banshee sap1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 287 |
| ✓ | BbhWordSorting127CTSort the following words alphabetically: List: whisper eyesight titmice mirage fletch cry biennial creedal miltonic fraudulent j1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT1 240 |
| ✓ | BbhWordSorting128CTSort the following words alphabetically: List: multitudinous adopt wage worcestershire pacifist afghan glimmer friday1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT997 |
| ✓ | BbhWordSorting129CTSort the following words alphabetically: List: lethe bilk perturb tactual1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT406 |
| ✓ | BbhWordSorting130CTSort the following words alphabetically: List: liggett sterno pintail dilate frolic piggish forth parameter slaughterhouse lacunae mcintosh bloodstain lazy unesco protector beachhead algonquin1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 362 |
| ✓ | BbhWordSorting131CTSort the following words alphabetically: List: erudite marriageable stahl coach inadvisable vicksburg virgo brownian eosine magnesium flax1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 137 |
| ✓ | BbhWordSorting132CTSort the following words alphabetically: List: clifford brindle siskin gloat florist triploidy willard sacramento1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 091 |
| ✓ | BbhWordSorting133CTSort the following words alphabetically: List: confrontation quantitative proserpine proofread daddy hirsute1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT1 019 |
| ✓ | BbhWordSorting134CTSort the following words alphabetically: List: allyn carbonaceous majorca investigatory supersede cetacean pathogenic johann tung paradigmatic pray1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT1 056 |
| ✓ | BbhWordSorting135CTSort the following words alphabetically: List: calyx emphases vii elgin perquisite implacable blackbird afro jura mayapple whit blame1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 041 |
| ✓ | BbhWordSorting136CTSort the following words alphabetically: List: cast palazzi inhibit arragon tabletop danbury abbe adposition emplace gavin income falsetto onerous1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 143 |
| ✓ | BbhWordSorting137CTSort the following words alphabetically: List: ulcerate ballroom sparse intrepid arcing prorogue dodge benefit coupon ladle buggy shagging bassoon hypothermia decide affirmative airframe junior nineveh schmitt1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT2 139 |
| ✓ | BbhWordSorting138CTSort the following words alphabetically: List: statuette whiteface lauderdale encumber atonic specify olivia adipic neutrino antique catch persona athlete sovereignty1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 271 |
| ✓ | BbhWordSorting139CTSort the following words alphabetically: List: hyperboloidal borough1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT376 |
| ✓ | BbhWordSorting140CTSort the following words alphabetically: List: yves butt scribe seaworthy manumitted clapeyron lift blackguard reconnaissance agile woodcut lightfooted sawbelly meager geophysical wiseacre mathieu cognoscenti purposive flamboyant1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 005 OUT1 940 |
| ✓ | BbhWordSorting141CTSort the following words alphabetically: List: duane harriet impressible whippet glamour cowbell vietnamese fest switchboard anaglyph texture1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT799 |
| ✓ | BbhWordSorting142CTSort the following words alphabetically: List: plural dose allstate stalin dyad multitudinous powderpuff1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT621 |
| ✓ | BbhWordSorting143CTSort the following words alphabetically: List: crossbill bushland capacitive meager ricochet battery enigma wysiwyg wallet lipton jane wacke contingent1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 144 |
Reasoning - BBH word sorting (set 5)
24 passed3 failed |
88.9% |
IN26 493 OUT29 566 | |
| ✓ | BbhWordSorting144CTSort the following words alphabetically: List: corpulent diagnose code scalp bombproof fluorine blythe swipe honeybee damn pore maharaja cytolysis solicit1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 791 |
| ✓ | BbhWordSorting145CTSort the following words alphabetically: List: fugue chicanery mountain1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT417 |
| ✓ | BbhWordSorting146CTSort the following words alphabetically: List: sideboard procrustean seepage buxton olympic tty typescript marque callus unitary cameron contribute precise verify shelf extensible methanol1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT2 211 |
| ✓ | BbhWordSorting147CTSort the following words alphabetically: List: sociolinguistic nigeria odysseus demystify proton voltaire penny contaminate bighorn stirrup1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 128 |
| ✓ | BbhWordSorting148CTSort the following words alphabetically: List: greasy lapidary mark1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT266 |
| ✓ | BbhWordSorting149CTSort the following words alphabetically: List: choreograph retardant multifarious pro carbonic equestrian wishful wigwam countersink libya authenticate metal have corvallis nitric petition obfuscatory1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 484 |
| × | BbhWordSorting150CTSort the following words alphabetically: List: oakland seaborg jacobi membrane trapezoidal allis marmot toggle anthology1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("allis anthology jacobi marmot membrane oakland seaborg toggle trapezoidal")
Context: BbhWordSorting150CT: expected sorted list, got [allis anthology jacobi membrane marmot oakland seaborg toggle trapezoidal] |
Failed | IN980 OUT639 |
| ✓ | BbhWordSorting151CTSort the following words alphabetically: List: monotonous slick provocation functor hideaway holeable assimilable lola calypso nebuchadnezzar dogwood pacifism bivariate bought1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 275 |
| ✓ | BbhWordSorting152CTSort the following words alphabetically: List: livre substitution ix frisian panoramic buckley1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT1 062 |
| ✓ | BbhWordSorting153CTSort the following words alphabetically: List: no spokane cartilaginous that'd science1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT826 |
| ✓ | BbhWordSorting154CTSort the following words alphabetically: List: splashy rectitude novelty1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT533 |
| ✓ | BbhWordSorting155CTSort the following words alphabetically: List: sediment varsity towel fracture1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT433 |
| ✓ | BbhWordSorting156CTSort the following words alphabetically: List: confidential ursa fiction heterozygous faery overture horehound1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 421 |
| ✓ | BbhWordSorting157CTSort the following words alphabetically: List: mckinley pesticide school radiometer marque prig relish trianon inspiration artful cancelled castrate glaucous ear croon tioga rothschild excite endpoint citadel1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 507 |
| ✓ | BbhWordSorting158CTSort the following words alphabetically: List: iniquity yoghurt behind volcanism hornpipe mcconnell scorn zip toroidal mollie sandy wellwisher inmate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT2 021 |
| ✓ | BbhWordSorting159CTSort the following words alphabetically: List: extempore gotten1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT562 |
| ✓ | BbhWordSorting160CTSort the following words alphabetically: List: yang relinquish merry maladjust olefin canaan ranch batavia1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT777 |
| ✓ | BbhWordSorting161CTSort the following words alphabetically: List: yuh edt cheddar oblivion from poignant pang1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT669 |
| ✓ | BbhWordSorting162CTSort the following words alphabetically: List: zan natalie coplanar stevenson1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT390 |
| ✓ | BbhWordSorting163CTSort the following words alphabetically: List: bottle kenney chapati okinawa boon bedtime1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT655 |
| ✓ | BbhWordSorting164CTSort the following words alphabetically: List: contrariety hoop muong dimorphic crochet forthwith hurty titrate prodigy polytope guaranteeing atavism emanate iniquity grind more katie1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 420 |
| × | BbhWordSorting165CTSort the following words alphabetically: List: sloane molten equip hillman max cochran practise advisee kuwait chancery hoofprint cardigan evildoer retinue wuhan ape apostate acrobacy crowbait1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("acrobacy advisee ape apostate cardigan chancery cochran crowbait equip evildoer hillman hoofprint kuwait max molten practise retinue sloane wuhan")
Context: BbhWordSorting165CT: expected sorted list, got [acrobacy advisee ape apostate cardigan chancery cochran crowbait evildoer equip hillman hoofprint kuwait max molten practise retinue sloane wuhan] |
Failed | IN1 000 OUT1 507 |
| × | BbhWordSorting166CTSort the following words alphabetically: List: catawba almagest transfusable milt photolysis limpid gnome archenemy sagging councilwomen decrement jungian1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("almagest archenemy catawba councilwomen decrement gnome jungian limpid milt photolysis sagging transfusable")
Context: BbhWordSorting166CT: expected sorted list, got [almagest archenemy councilwomen catawba decrement gnome jungian limpid milt photolysis sagging transfusable] |
Failed | IN987 OUT843 |
| ✓ | BbhWordSorting167CTSort the following words alphabetically: List: inscribe upraise perceptive tragedian peculate cunard posterior crude danubian1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT957 |
| ✓ | BbhWordSorting168CTSort the following words alphabetically: List: madeira offprint trafficked immemorial subject micro clinic sprue marx va anaheim eaten1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT979 |
| ✓ | BbhWordSorting169CTSort the following words alphabetically: List: reedy bully multiple edt sell referring lexicographer tit flogging perceptive minor bosporus rancorous forfeit pizza pungent sedition cork1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 654 |
| ✓ | BbhWordSorting170CTSort the following words alphabetically: List: thirteenth tinfoil thimble snuff ernie pow celebrity abuilding indiscoverable chigger burgher synchronous yap medlar exorcism newsboy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT2 139 |
Reasoning - BBH word sorting (set 6)
23 passed4 failed |
85.2% |
IN26 640 OUT31 915 | |
| ✓ | BbhWordSorting171CTSort the following words alphabetically: List: saprophytic confident pam stowaway stupor foundling compton1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 100 |
| ✓ | BbhWordSorting172CTSort the following words alphabetically: List: epiphyte swift sail anchor barre dimension faunal integrate eleanor edgy singular patrician orthodoxy parasol pendant buckle concatenate masochist1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT1 057 |
| ✓ | BbhWordSorting173CTSort the following words alphabetically: List: butterscotch veterinarian kochab molybdate extoller amos farrell academia glance predatory beautiful ford fiducial metzler europium circuitous diatom monomer1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 280 |
| ✓ | BbhWordSorting174CTSort the following words alphabetically: List: mater rica sigmund dean eosine inebriate gibson vassar mulligatawny formula1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 124 |
| ✓ | BbhWordSorting175CTSort the following words alphabetically: List: shakespearian laudatory1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT224 |
| ✓ | BbhWordSorting176CTSort the following words alphabetically: List: fasciculate judicature presto1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT506 |
| ✓ | BbhWordSorting177CTSort the following words alphabetically: List: thereupon wireman crag scrawny electrolytic wife cutover dickson clytemnestra took diocletian lipton inhuman stalk workplace marginal1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT996 |
| × | BbhWordSorting178CTSort the following words alphabetically: List: bruckner admiration technion bandgap heron postprocessor maidenhair fungus allusion gambit accessory proscenium cruise1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("accessory admiration allusion bandgap bruckner cruise fungus gambit heron maidenhair postprocessor proscenium technion")
Context: BbhWordSorting178CT: expected sorted list, got [accessory allusion admiration bandgap bruckner cruise fungus gambit heron maidenhair postprocessor proscenium technion] |
Failed | IN988 OUT1 014 |
| ✓ | BbhWordSorting179CTSort the following words alphabetically: List: grammarian chivalry polkadot gastronomic ferocity brenda knutson endothelial phenotype irreducible bilinear rockaway derivate eaten cacao scurrilous1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 133 |
| ✓ | BbhWordSorting180CTSort the following words alphabetically: List: hyperboloid buckle liquidate alkali nagasaki breach falsetto parmesan mirth1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 023 |
| × | BbhWordSorting181CTSort the following words alphabetically: List: hornblower dissipate amanita canticle annoy besiege straight notre propylene sepia california pasture encephalitis boggle crocodilian dexter snipe amatory dizzy psychiatric1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("amanita amatory annoy besiege boggle california canticle crocodilian dexter dissipate dizzy encephalitis hornblower notre pasture propylene psychiatric sepia snipe straight")
Context: BbhWordSorting181CT: expected sorted list, got [annoy amanita amatory besiege boggle california canticle crocodilian dexter dissipate dizzy encephalitis hornblower notre pasture propylene psychiatric sepia snipe straight] |
Failed | IN999 OUT1 782 |
| ✓ | BbhWordSorting182CTSort the following words alphabetically: List: peaceable residue sulfanilamide kingsley eft caliber headboard stamp phillip upholster irishman lobby nary gastrointestinal fragile faulkner capricious ouzo phylum1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 006 |
| ✓ | BbhWordSorting183CTSort the following words alphabetically: List: multiplet catsup hypothalamus scherzo bridesmaid average therewith invocable coddle charm inequity region abbas trumpery memorandum shutout integral dogfish phloem inconvertible1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 003 OUT1 954 |
| ✓ | BbhWordSorting184CTSort the following words alphabetically: List: windowsill appoint biharmonic moustache baneberry wiry dyne pirate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT1 231 |
| ✓ | BbhWordSorting185CTSort the following words alphabetically: List: zoroaster skimpy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT392 |
| × | BbhWordSorting186CTSort the following words alphabetically: List: almost delphic semitic unbeknownst astute deadlock libertarian antic levitate execution sanction scathe storehouse dandelion affable sweeney fortunate leverage horntail apache1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("affable almost antic apache astute dandelion deadlock delphic execution fortunate horntail leverage levitate libertarian sanction scathe semitic storehouse sweeney unbeknownst")
Context: BbhWordSorting186CT: expected sorted list, got [affable almost apache antic astute dandelion deadlock delphic execution fortunate horntail libertarian leverage levitate sanction scathe semitic storehouse sweeney unbeknownst] |
Failed | IN997 OUT2 008 |
| ✓ | BbhWordSorting187CTSort the following words alphabetically: List: percent maser roadside we're caching distort phillip emboss epistemology gherkin somewhat spicy injustice winston savoyard delicious defend indicate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 836 |
| ✓ | BbhWordSorting188CTSort the following words alphabetically: List: anarchic liven shortcut total diatom guelph usage acoustic yarmulke phenomenology neo bureaucracy trammel leftward immovable fabricate syndrome suggestive provide1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT1 060 |
| × | BbhWordSorting189CTSort the following words alphabetically: List: them abdominal address fomalhaut pay optimal hanoverian yang berry tattoo islamabad swathe jordan bounty stearic stigmata involve effusive tornado1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("abdominal address berry bounty effusive fomalhaut hanoverian involve islamabad jordan optimal pay stearic stigmata swathe tattoo them tornado yang")
Context: BbhWordSorting189CT: expected sorted list, got [abdominal address berry bounty effusive fomalhaut hanoverian involve islamabad jordan optimal pay stearic stigmata swathе tattoo them tornado yang] |
Failed | IN993 OUT2 009 |
| ✓ | BbhWordSorting190CTSort the following words alphabetically: List: maraud downtrodden hurst propane morphine parsonage gadgetry inertial gamin1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 160 |
| ✓ | BbhWordSorting191CTSort the following words alphabetically: List: foamflower lawful cayenne chandigarh only excursion declassify equanimity aseptic inclement october debauch kruger groupoid scorch dingy alterate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 497 |
| ✓ | BbhWordSorting192CTSort the following words alphabetically: List: acidify antagonism asteria1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT651 |
| ✓ | BbhWordSorting193CTSort the following words alphabetically: List: elton olympic iodinate captious ineligible sherman1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT1 009 |
| ✓ | BbhWordSorting194CTSort the following words alphabetically: List: alleviate duopoly mattress gland benelux townsmen buoyant klaxon hardbound tomography felice gunk1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 252 |
| ✓ | BbhWordSorting195CTSort the following words alphabetically: List: solvate nat usable kajar downey vision detest aitken barycentric1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT898 |
| ✓ | BbhWordSorting196CTSort the following words alphabetically: List: griffith common nanosecond loosestrife decoy swabby aerospace denmark yates frill spitz enviable screechy allure sow spermatozoa saute jibe exclusive1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 001 OUT1 442 |
| ✓ | BbhWordSorting197CTSort the following words alphabetically: List: reverie giantess muddy mast callous bate dnieper prank cortez staunch satisfy dogging moran climb garrison1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 271 |
Reasoning - BBH word sorting (set 7)
24 passed2 failed |
92.3% |
IN25 546 OUT28 725 | |
| ✓ | BbhWordSorting198CTSort the following words alphabetically: List: avoid whimsey grassland elegy pitfall error gaillardia hostile caramel conclave stepchild accept carbuncle compressor utopia drib spectra rosa embower1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 065 |
| ✓ | BbhWordSorting199CTSort the following words alphabetically: List: rowley midshipman saline furl geld locomote exhumation anarchic bstj gradual riddance j liniment pantheist elution profess1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 279 |
| ✓ | BbhWordSorting200CTSort the following words alphabetically: List: hangout drafty supposable sushi proscenium anticonvulsant you'd magnificent scrub embolden hasty pewee carrageen disseminate glamour discovery registrar acuity1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT2 380 |
| ✓ | BbhWordSorting201CTSort the following words alphabetically: List: sweatband murray1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT202 |
| ✓ | BbhWordSorting202CTSort the following words alphabetically: List: resignation dormouse bathos walt amethyst obtuse1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT651 |
| ✓ | BbhWordSorting203CTSort the following words alphabetically: List: paginate amicable they've histology extreme indoeuropean sonority tramway turvy coo deerstalker browne henchman pelvis calumny1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT1 233 |
| ✓ | BbhWordSorting204CTSort the following words alphabetically: List: transgressor sidewinder tacky casualty shouldn't courtier leprosy avoidance gibbon merge1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 581 |
| ✓ | BbhWordSorting205CTSort the following words alphabetically: List: intercept tamp geopolitic loci newsmen umber credit atmospheric lunge siren swart chess1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT1 163 |
| ✓ | BbhWordSorting206CTSort the following words alphabetically: List: wink envious scotia planetaria pooh emancipate army1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT742 |
| ✓ | BbhWordSorting207CTSort the following words alphabetically: List: condescend rampant percolate coltish date rochester placid significant1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 334 |
| ✓ | BbhWordSorting208CTSort the following words alphabetically: List: infantrymen knob yaounde aqueous deregulate lysergic gala1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT710 |
| ✓ | BbhWordSorting209CTSort the following words alphabetically: List: promulgate altercate foraminifera sophocles raft wrongdoer syllabus jive cornerstone gossamer courtroom insist dusenberg sal1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT1 227 |
| ✓ | BbhWordSorting210CTSort the following words alphabetically: List: jacob sleight globule consonant musician1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT319 |
| ✓ | BbhWordSorting211CTSort the following words alphabetically: List: nascent muddy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN965 OUT502 |
| ✓ | BbhWordSorting212CTSort the following words alphabetically: List: automate abner shampoo gurgle none exquisite abramson amity fruitful shorten waterproof1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 112 |
| ✓ | BbhWordSorting213CTSort the following words alphabetically: List: search marcello depreciate unary frame numbly sprout clench agamemnon herbivorous eject zaire lien forum1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT1 112 |
| × | BbhWordSorting214CTSort the following words alphabetically: List: saul fresco teen chopstick footwork smalley anaplasmosis palisade orthography storey bumble pilate clue ingot fiesta1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("anaplasmosis bumble chopstick clue fiesta footwork fresco ingot orthography palisade pilate saul smalley storey teen")
Context: BbhWordSorting214CT: expected sorted list, got [anaplasmosis bumble chopstick clue fiesta fresco footwork ingot orthography palisade pilate saul smalley storey teen] |
Failed | IN992 OUT1 261 |
| ✓ | BbhWordSorting215CTSort the following words alphabetically: List: avalanche chaplin squabble coachman ipsilateral clonic cameroon fortran cram medea referring kennan canal pyridine ussr postpone1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT1 777 |
| × | BbhWordSorting216CTSort the following words alphabetically: List: peculiar margaret contravention ingratiate dreg superintendent drapery bizarre sequential1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("bizarre contravention drapery dreg ingratiate margaret peculiar sequential superintendent")
Context: BbhWordSorting216CT: expected sorted list, got [bizarre contravention drapery dreg ingrati ate margaret peculiar sequential superintendent] |
Failed | IN979 OUT1 517 |
| ✓ | BbhWordSorting217CTSort the following words alphabetically: List: sweatshirt bone ia wagoneer doleful homeobox hindustan convergent1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT875 |
| ✓ | BbhWordSorting218CTSort the following words alphabetically: List: pewter gross gujarati cortland diffusible nomograph stag broadcast galvanometer transition incestuous sketchbook scout larynx1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 165 |
| ✓ | BbhWordSorting219CTSort the following words alphabetically: List: portsmouth impalpable portrait gouda brainy succumb fatuous hoof lisbon erudite malaria enigma isaacson cony stronghold servomechanism1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 513 |
| ✓ | BbhWordSorting220CTSort the following words alphabetically: List: testate orchestra permafrost leasehold shiva1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT577 |
| ✓ | BbhWordSorting221CTSort the following words alphabetically: List: logarithm robertson inanimate uganda micron incurring wilcox lumber sammy souvenir newman profuse maladapt cone butch airlift homeowner1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT1 860 |
| ✓ | BbhWordSorting222CTSort the following words alphabetically: List: chrysalis wallaby1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT429 |
| ✓ | BbhWordSorting223CTSort the following words alphabetically: List: dredge checksum huckster baronial spotlight circumstance eulogy comment felicia dartmouth monochromator neuroanatomic emittance1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT1 139 |
Reasoning - BBH word sorting (set 8)
24 passed2 failed |
92.3% |
IN25 604 OUT33 344 | |
| ✓ | BbhWordSorting224CTSort the following words alphabetically: List: breezy germinal biplane canada nutrient archipelago peloponnese thigh conspiracy constructor accrue dobbin lost hamburger insubstantial malleable ted laramie1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 649 |
| ✓ | BbhWordSorting225CTSort the following words alphabetically: List: wagging cabdriver astronomic pivot loch coherent1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT838 |
| ✓ | BbhWordSorting226CTSort the following words alphabetically: List: inability nucleus vague possible invocable janice erg1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT1 268 |
| × | BbhWordSorting227CTSort the following words alphabetically: List: gag grecian ada nobody militia wildcat right abc blend cankerworm hanukkah pierson ron falcon quad flamboyant lobster kruger austere indicate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("abc ada austere blend cankerworm falcon flamboyant gag grecian hanukkah indicate kruger lobster militia nobody pierson quad right ron wildcat")
Context: BbhWordSorting227CT: expected sorted list, got [abc austere ada blend cankerworm falcon flamboyant gag grecian hanukkah indicate kruger lobster militia nobody pierson quad right ron wildcat] |
Failed | IN994 OUT1 714 |
| ✓ | BbhWordSorting228CTSort the following words alphabetically: List: babysat curvaceous hugh cutaneous spoke consul stationarity regiment1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT944 |
| ✓ | BbhWordSorting229CTSort the following words alphabetically: List: woebegone nimble complementary afternoon immaculate laughlin kind hesitate loire septuagenarian sandia toggle dixie horsepower sierra shuffleboard mechanism1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT2 350 |
| ✓ | BbhWordSorting230CTSort the following words alphabetically: List: linchpin arrear eavesdropping fasciculate shin postdoctoral maritime ibn henry hermaphrodite sticky jane brookside incorrigible vehicular herodotus1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 001 OUT1 412 |
| ✓ | BbhWordSorting231CTSort the following words alphabetically: List: sinter cortex kangaroo pillage threshold insane mcleod roundabout stipulate marionette trammel incident1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 603 |
| ✓ | BbhWordSorting232CTSort the following words alphabetically: List: soulful dnieper labile lease vehicular1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT764 |
| ✓ | BbhWordSorting233CTSort the following words alphabetically: List: cornerstone green brindle prowl nonce ballard scrappy credulous des pitney difluoride toward predilect nostalgic rape mixup horseplay jew curio1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 793 |
| ✓ | BbhWordSorting234CTSort the following words alphabetically: List: covenant thence davenport sonorant scoreboard densitometer noisy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT811 |
| ✓ | BbhWordSorting235CTSort the following words alphabetically: List: inhabitation bona cicero purport gaussian silver responsible rastus zig exonerate portfolio coastal asset ruanda dusky handlebar1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 238 |
| ✓ | BbhWordSorting236CTSort the following words alphabetically: List: implementor bangui nudge cavalier refractory islamabad contextual abramson waite railway carlisle magistrate silvery picnicking dustbin emacs1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 508 |
| ✓ | BbhWordSorting237CTSort the following words alphabetically: List: strove hendrickson apostasy jacm indonesia vibrate afloat lanthanide grosbeak scavenge quark chattel conner bechtel melancholy ferment1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT1 856 |
| ✓ | BbhWordSorting238CTSort the following words alphabetically: List: spongy separate gabriel jackson quail hendrickson washy backslide shelter calvert anastomosis variety commando pizzicato sticktight syndicate hollister1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 716 |
| ✓ | BbhWordSorting239CTSort the following words alphabetically: List: behold oxalic maybe hew steel termcap pray stiffen dissipate misogyny format dew1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 165 |
| ✓ | BbhWordSorting240CTSort the following words alphabetically: List: dove rodriguez sonant monaco strap horticulture claret play paschal bizarre valuate colonist baden deplore1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 020 |
| × | BbhWordSorting241CTSort the following words alphabetically: List: patrolman arabesque novel spearmint arkansan bend brassiere diagnosable ytterbium sheep bedroom yen planeload deterrent fossiliferous alveolar trident fluke curvilinear1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("alveolar arabesque arkansan bedroom bend brassiere curvilinear deterrent diagnosable fluke fossiliferous novel patrolman planeload sheep spearmint trident yen ytterbium")
Context: BbhWordSorting241CT: expected sorted list, got [alveolar arabesque arkansan bend bedroom brassiere curvilinear deterrent diagnosable fluke fossiliferous novel patrolman planeload sheep spearmint trident ytterbium yen] |
Failed | IN1 002 OUT1 400 |
| ✓ | BbhWordSorting242CTSort the following words alphabetically: List: papaw perfidious frugal labial feel sonar blunderbuss box dinnertime oresteia1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 032 |
| ✓ | BbhWordSorting243CTSort the following words alphabetically: List: haddock colossus sheepskin honeymoon candid pragmatic agrarian people applicate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 006 |
| ✓ | BbhWordSorting244CTSort the following words alphabetically: List: catechism daddy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT621 |
| ✓ | BbhWordSorting245CTSort the following words alphabetically: List: holocene muriel r&d pathetic talon mare staircase bradshaw aperture sigh1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT1 111 |
| ✓ | BbhWordSorting246CTSort the following words alphabetically: List: bust midshipmen musicale urinary chalk rocket resiny horror tangential sapient dumb serf pathway urea goucher fatty masonry dentistry cowboy roadrunner1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 892 |
| ✓ | BbhWordSorting247CTSort the following words alphabetically: List: sanhedrin scratchy helical beau venezuela awash bessie extricable indoeuropean vice pendulum cream animism1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 262 |
| ✓ | BbhWordSorting248CTSort the following words alphabetically: List: him correspond seashore herpes1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT488 |
| ✓ | BbhWordSorting249CTSort the following words alphabetically: List: worcestershire bonito fritter dreamboat haggard nose whodunit1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT883 |