Benchmark summary
BBH-based reasoning benchmark ported to the SC# evaluation runtime. It tests multi-step reasoning across challenging tasks involving logic, deduction, symbolic manipulation, planning, and problem solving.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Reasoning - BBH boolean expressions (set 1)
32 passed |
100.0% |
IN30 784 OUT29 454 | |
| ✓ | BbhBooleanExpressionsBD1not True or ( False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT678 |
| ✓ | BbhBooleanExpressionsBD10( True and not True and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 014 |
| ✓ | BbhBooleanExpressionsBD11False and False and False or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 368 |
| ✓ | BbhBooleanExpressionsBD12False or ( False and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT709 |
| ✓ | BbhBooleanExpressionsBD13True and False or ( not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT867 |
| ✓ | BbhBooleanExpressionsBD14not not ( True ) and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT966 |
| ✓ | BbhBooleanExpressionsBD15not False or ( True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT842 |
| ✓ | BbhBooleanExpressionsBD16not ( True ) or False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT932 |
| ✓ | BbhBooleanExpressionsBD17( True and not not not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT474 |
| ✓ | BbhBooleanExpressionsBD18( False or not False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT549 |
| ✓ | BbhBooleanExpressionsBD19False and False or True and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT951 |
| ✓ | BbhBooleanExpressionsBD2not True or ( True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT428 |
| ✓ | BbhBooleanExpressionsBD20not not False or not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT665 |
| ✓ | BbhBooleanExpressionsBD21True and not True or False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 141 |
| ✓ | BbhBooleanExpressionsBD22not False or True and False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT648 |
| ✓ | BbhBooleanExpressionsBD23False and ( not False ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT515 |
| ✓ | BbhBooleanExpressionsBD24not not not not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT832 |
| ✓ | BbhBooleanExpressionsBD25( True ) and True or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT892 |
| ✓ | BbhBooleanExpressionsBD26not True or False and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT807 |
| ✓ | BbhBooleanExpressionsBD27not not not not not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT808 |
| ✓ | BbhBooleanExpressionsBD28not True or not False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT917 |
| ✓ | BbhBooleanExpressionsBD29False or True or False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT685 |
| ✓ | BbhBooleanExpressionsBD3False or ( False ) and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 477 |
| ✓ | BbhBooleanExpressionsBD30not ( True or True ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT862 |
| ✓ | BbhBooleanExpressionsBD31not not not not True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 261 |
| ✓ | BbhBooleanExpressionsBD32( False ) or False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT908 |
| ✓ | BbhBooleanExpressionsBD4not False or True and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT3 404 |
| ✓ | BbhBooleanExpressionsBD5not True or False or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT668 |
| ✓ | BbhBooleanExpressionsBD6True and True and False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT958 |
| ✓ | BbhBooleanExpressionsBD7not not True and not True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT805 |
| ✓ | BbhBooleanExpressionsBD8not not not ( True and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT467 |
| ✓ | BbhBooleanExpressionsBD9not not False and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT956 |
Reasoning - BBH boolean expressions (set 2)
31 passed1 failed |
96.9% |
IN30 784 OUT25 804 | |
| ✓ | BbhBooleanExpressionsBD33( False and True or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 006 |
| ✓ | BbhBooleanExpressionsBD34not not not ( False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT642 |
| ✓ | BbhBooleanExpressionsBD35not not True and not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 165 |
| ✓ | BbhBooleanExpressionsBD36not False or True and not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 026 |
| ✓ | BbhBooleanExpressionsBD37True or True or not False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT791 |
| ✓ | BbhBooleanExpressionsBD38( not False and False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT965 |
| ✓ | BbhBooleanExpressionsBD39( ( not True ) or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT284 |
| ✓ | BbhBooleanExpressionsBD40not not ( not False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT730 |
| ✓ | BbhBooleanExpressionsBD41not True or False or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT804 |
| ✓ | BbhBooleanExpressionsBD42False or ( not False ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT589 |
| ✓ | BbhBooleanExpressionsBD43not False or False or ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT656 |
| ✓ | BbhBooleanExpressionsBD44not not False or False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 187 |
| ✓ | BbhBooleanExpressionsBD45True or ( ( not True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT469 |
| ✓ | BbhBooleanExpressionsBD46not False and not False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT320 |
| ✓ | BbhBooleanExpressionsBD47( False ) and not False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 046 |
| ✓ | BbhBooleanExpressionsBD48( True ) and False or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT887 |
| ✓ | BbhBooleanExpressionsBD49( not False and ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT429 |
| ✓ | BbhBooleanExpressionsBD50False or not ( False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT394 |
| ✓ | BbhBooleanExpressionsBD51True and not False and not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT802 |
| ✓ | BbhBooleanExpressionsBD52not True and True and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 200 |
| ✓ | BbhBooleanExpressionsBD53not not ( False and not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT585 |
| × | BbhBooleanExpressionsBD54True or not ( ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("True")
Context: BbhBooleanExpressionsBD54: expected True, got [false]
Values:
got = "false" |
Failed | IN962 OUT1 208 |
| ✓ | BbhBooleanExpressionsBD55False or not True or False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT672 |
| ✓ | BbhBooleanExpressionsBD56False or not not True or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 274 |
| ✓ | BbhBooleanExpressionsBD57not ( True or True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT540 |
| ✓ | BbhBooleanExpressionsBD58False or not not not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT907 |
| ✓ | BbhBooleanExpressionsBD59not ( False ) or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT709 |
| ✓ | BbhBooleanExpressionsBD60True and not False or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 210 |
| ✓ | BbhBooleanExpressionsBD61True or ( not ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT395 |
| ✓ | BbhBooleanExpressionsBD62( False or True and not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT685 |
| ✓ | BbhBooleanExpressionsBD63False or not True and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 745 |
| ✓ | BbhBooleanExpressionsBD64not not not not not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT482 |
Reasoning - BBH boolean expressions (set 3)
31 passed1 failed |
96.9% |
IN30 784 OUT27 388 | |
| ✓ | BbhBooleanExpressionsBD65( not False ) and False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT966 |
| ✓ | BbhBooleanExpressionsBD66False and False or not ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 437 |
| ✓ | BbhBooleanExpressionsBD67( not False or True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT482 |
| ✓ | BbhBooleanExpressionsBD68not ( False ) and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT714 |
| ✓ | BbhBooleanExpressionsBD69False or True and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT631 |
| ✓ | BbhBooleanExpressionsBD70( True ) and True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT405 |
| ✓ | BbhBooleanExpressionsBD71( not True ) and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT526 |
| ✓ | BbhBooleanExpressionsBD72( not not True and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT496 |
| ✓ | BbhBooleanExpressionsBD73True and ( False ) or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT985 |
| ✓ | BbhBooleanExpressionsBD74not not not False or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT2 134 |
| ✓ | BbhBooleanExpressionsBD75not not not False and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT647 |
| ✓ | BbhBooleanExpressionsBD76True or ( True or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT624 |
| ✓ | BbhBooleanExpressionsBD77not False or ( ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT705 |
| ✓ | BbhBooleanExpressionsBD78not not ( False and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT941 |
| ✓ | BbhBooleanExpressionsBD79not not not not True or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 019 |
| ✓ | BbhBooleanExpressionsBD80False and not ( False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT353 |
| ✓ | BbhBooleanExpressionsBD81True and True and True or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 074 |
| ✓ | BbhBooleanExpressionsBD82False or not False or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 376 |
| ✓ | BbhBooleanExpressionsBD83not False and False and False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 017 |
| ✓ | BbhBooleanExpressionsBD84( False ) or not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT607 |
| ✓ | BbhBooleanExpressionsBD85( not False ) or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT271 |
| ✓ | BbhBooleanExpressionsBD86not not ( False or not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT526 |
| ✓ | BbhBooleanExpressionsBD87False or not False or False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT605 |
| ✓ | BbhBooleanExpressionsBD88False or not False and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT898 |
| ✓ | BbhBooleanExpressionsBD89False or ( False ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT645 |
| ✓ | BbhBooleanExpressionsBD90not True or False and True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 106 |
| ✓ | BbhBooleanExpressionsBD91True or ( True or not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT761 |
| ✓ | BbhBooleanExpressionsBD92True and not False and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT2 686 |
| ✓ | BbhBooleanExpressionsBD93False or not False or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT489 |
| ✓ | BbhBooleanExpressionsBD94( True or not False ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT690 |
| × | BbhBooleanExpressionsBD95False or not ( True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("False")
Context: BbhBooleanExpressionsBD95: expected False, got [true]
Values:
got = "true" |
Failed | IN962 OUT969 |
| ✓ | BbhBooleanExpressionsBD96not False and not True and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT603 |
Reasoning - BBH boolean expressions (set 4)
32 passed |
100.0% |
IN30 784 OUT25 835 | |
| ✓ | BbhBooleanExpressionsBD100( True or not not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT392 |
| ✓ | BbhBooleanExpressionsBD101False and ( False or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT813 |
| ✓ | BbhBooleanExpressionsBD102False or not True or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 572 |
| ✓ | BbhBooleanExpressionsBD103True and not True or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT828 |
| ✓ | BbhBooleanExpressionsBD104( not False or ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT539 |
| ✓ | BbhBooleanExpressionsBD105( ( not ( True ) ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT506 |
| ✓ | BbhBooleanExpressionsBD106False and False and not ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 009 |
| ✓ | BbhBooleanExpressionsBD107not True and not True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT395 |
| ✓ | BbhBooleanExpressionsBD108not ( True or False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT435 |
| ✓ | BbhBooleanExpressionsBD109False or ( True ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT823 |
| ✓ | BbhBooleanExpressionsBD110False or False and False or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 806 |
| ✓ | BbhBooleanExpressionsBD111True or False or not True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 458 |
| ✓ | BbhBooleanExpressionsBD112not True and ( True and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT469 |
| ✓ | BbhBooleanExpressionsBD113not True and True or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 160 |
| ✓ | BbhBooleanExpressionsBD114not False or ( True or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT396 |
| ✓ | BbhBooleanExpressionsBD115not ( False or ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT599 |
| ✓ | BbhBooleanExpressionsBD116not ( True or True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT610 |
| ✓ | BbhBooleanExpressionsBD117False or not False or True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT583 |
| ✓ | BbhBooleanExpressionsBD118False or False and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT2 217 |
| ✓ | BbhBooleanExpressionsBD119not ( True or False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT323 |
| ✓ | BbhBooleanExpressionsBD120( False ) or not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 085 |
| ✓ | BbhBooleanExpressionsBD121True and True and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT838 |
| ✓ | BbhBooleanExpressionsBD122( True ) or True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT881 |
| ✓ | BbhBooleanExpressionsBD123False and False and not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT649 |
| ✓ | BbhBooleanExpressionsBD124( True ) and not True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT534 |
| ✓ | BbhBooleanExpressionsBD125not not not True and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT698 |
| ✓ | BbhBooleanExpressionsBD126False and False and not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 068 |
| ✓ | BbhBooleanExpressionsBD127False and False or False or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT747 |
| ✓ | BbhBooleanExpressionsBD128False or ( not ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT371 |
| ✓ | BbhBooleanExpressionsBD97False and ( True and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT591 |
| ✓ | BbhBooleanExpressionsBD98not False or ( False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT844 |
| ✓ | BbhBooleanExpressionsBD99False and not ( False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT596 |
Reasoning - BBH boolean expressions (set 5)
30 passed |
100.0% |
IN28 860 OUT22 029 | |
| ✓ | BbhBooleanExpressionsBD129not True and True and not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT870 |
| ✓ | BbhBooleanExpressionsBD130( False ) and ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT758 |
| ✓ | BbhBooleanExpressionsBD131( not True ) and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT552 |
| ✓ | BbhBooleanExpressionsBD132False and True and ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT552 |
| ✓ | BbhBooleanExpressionsBD133( ( True ) ) or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT530 |
| ✓ | BbhBooleanExpressionsBD134not True and False or False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT812 |
| ✓ | BbhBooleanExpressionsBD135True and not True and True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT516 |
| ✓ | BbhBooleanExpressionsBD136( True or True or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT633 |
| ✓ | BbhBooleanExpressionsBD137True or not False and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 332 |
| ✓ | BbhBooleanExpressionsBD138( not ( False ) or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT726 |
| ✓ | BbhBooleanExpressionsBD139True and not ( True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT385 |
| ✓ | BbhBooleanExpressionsBD140not ( False and True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT567 |
| ✓ | BbhBooleanExpressionsBD141not ( True and True ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT696 |
| ✓ | BbhBooleanExpressionsBD142False or not True and not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 183 |
| ✓ | BbhBooleanExpressionsBD143not True and False or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT749 |
| ✓ | BbhBooleanExpressionsBD144not True or True and False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT813 |
| ✓ | BbhBooleanExpressionsBD145( not ( False ) and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT420 |
| ✓ | BbhBooleanExpressionsBD146False or ( True and not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT298 |
| ✓ | BbhBooleanExpressionsBD147not False and not True or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT945 |
| ✓ | BbhBooleanExpressionsBD148( False or not not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT760 |
| ✓ | BbhBooleanExpressionsBD149( not False ) or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT928 |
| ✓ | BbhBooleanExpressionsBD150( not not True ) or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT370 |
| ✓ | BbhBooleanExpressionsBD151True or not ( ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT901 |
| ✓ | BbhBooleanExpressionsBD152( False ) and False or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 326 |
| ✓ | BbhBooleanExpressionsBD153not False and True or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT832 |
| ✓ | BbhBooleanExpressionsBD154True or not not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT658 |
| ✓ | BbhBooleanExpressionsBD155False and not True and False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT708 |
| ✓ | BbhBooleanExpressionsBD156not not True and True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT793 |
| ✓ | BbhBooleanExpressionsBD157not False or not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT569 |
| ✓ | BbhBooleanExpressionsBD158not not ( True ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT847 |
Reasoning - BBH boolean expressions (set 6)
31 passed1 failed |
96.9% |
IN30 784 OUT26 899 | |
| ✓ | BbhBooleanExpressionsBe166( True and not True and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT810 |
| ✓ | BbhBooleanExpressionsBe167True or False or not True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT892 |
| ✓ | BbhBooleanExpressionsBe168not True or True and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 227 |
| ✓ | BbhBooleanExpressionsBe169True and not not not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT422 |
| ✓ | BbhBooleanExpressionsBe170not True and ( False ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT758 |
| ✓ | BbhBooleanExpressionsBe171False and False or not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT904 |
| ✓ | BbhBooleanExpressionsBe172True or False and not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 221 |
| ✓ | BbhBooleanExpressionsBe173not ( False or False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT482 |
| ✓ | BbhBooleanExpressionsBe174not False or True or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 036 |
| ✓ | BbhBooleanExpressionsBe175not False and False or False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT697 |
| ✓ | BbhBooleanExpressionsBe176( not True ) and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT349 |
| ✓ | BbhBooleanExpressionsBe177True and not not False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 453 |
| ✓ | BbhBooleanExpressionsBe178not ( not not False ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT629 |
| ✓ | BbhBooleanExpressionsBe179not ( not False ) and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT718 |
| ✓ | BbhBooleanExpressionsBe180False or True and not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT999 |
| ✓ | BbhBooleanExpressionsBe181( not not not True and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT724 |
| ✓ | BbhBooleanExpressionsBe182False and not ( True ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT421 |
| ✓ | BbhBooleanExpressionsBe183( not True and False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT680 |
| ✓ | BbhBooleanExpressionsBe184not ( not not True ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 135 |
| ✓ | BbhBooleanExpressionsBe185( not False ) or ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT385 |
| ✓ | BbhBooleanExpressionsBe186not not not False and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 537 |
| ✓ | BbhBooleanExpressionsBe187True or not True or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT816 |
| ✓ | BbhBooleanExpressionsBe188not False or not not False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 199 |
| ✓ | BbhBooleanExpressionsBe189not False and ( False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT222 |
| ✓ | BbhBooleanExpressionsBe190False and True or ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT866 |
| ✓ | BbhBooleanExpressionsBe191False and not False and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT752 |
| ✓ | BbhBooleanExpressionsBe192not ( ( False and True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT416 |
| × | BbhBooleanExpressionsBe193False or ( not False ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("False")
Context: BbhBooleanExpressionsBe193: expected False, got [** False] |
Failed | IN962 OUT551 |
| ✓ | BbhBooleanExpressionsBe194False and not False or False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 302 |
| ✓ | BbhBooleanExpressionsBe195not False and not not True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT553 |
| ✓ | BbhBooleanExpressionsBe196True and True or False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 734 |
| ✓ | BbhBooleanExpressionsBe197not True or False and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 009 |
Reasoning - BBH boolean expressions (set 7)
32 passed |
100.0% |
IN30 784 OUT27 665 | |
| ✓ | BbhBooleanExpressionsBe198True or False or not True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 159 |
| ✓ | BbhBooleanExpressionsBe199False or False or not True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT574 |
| ✓ | BbhBooleanExpressionsBe200False or False or True and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT648 |
| ✓ | BbhBooleanExpressionsBe201( not ( True or False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT538 |
| ✓ | BbhBooleanExpressionsBe202not False and False or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT799 |
| ✓ | BbhBooleanExpressionsBe203True and False or ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT675 |
| ✓ | BbhBooleanExpressionsBe204not False or not not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT869 |
| ✓ | BbhBooleanExpressionsBe205True and ( not ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT330 |
| ✓ | BbhBooleanExpressionsBe206True and False and not True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 329 |
| ✓ | BbhBooleanExpressionsBe207( ( not False ) ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT570 |
| ✓ | BbhBooleanExpressionsBe208not ( True ) and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT664 |
| ✓ | BbhBooleanExpressionsBe209True or not False and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 274 |
| ✓ | BbhBooleanExpressionsBe210True and not ( False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT427 |
| ✓ | BbhBooleanExpressionsBe211not True and False or False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 392 |
| ✓ | BbhBooleanExpressionsBe212not True or not False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 060 |
| ✓ | BbhBooleanExpressionsBe213not True or False or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT2 263 |
| ✓ | BbhBooleanExpressionsBe214not True and True and False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT708 |
| ✓ | BbhBooleanExpressionsBe215False and False or not True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 203 |
| ✓ | BbhBooleanExpressionsBe216( not False or False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT641 |
| ✓ | BbhBooleanExpressionsBe217( True or False ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT515 |
| ✓ | BbhBooleanExpressionsBe218not not not ( False ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT746 |
| ✓ | BbhBooleanExpressionsBe219not False and False and False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT682 |
| ✓ | BbhBooleanExpressionsBe220False and not True and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT966 |
| ✓ | BbhBooleanExpressionsBe221( False or not False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT731 |
| ✓ | BbhBooleanExpressionsBe222False or ( False or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT477 |
| ✓ | BbhBooleanExpressionsBe223not False or True or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT952 |
| ✓ | BbhBooleanExpressionsBe224True or ( not False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT663 |
| ✓ | BbhBooleanExpressionsBe225not False and False or True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT794 |
| ✓ | BbhBooleanExpressionsBe226False and not ( not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 148 |
| ✓ | BbhBooleanExpressionsBe227not not ( not True ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT861 |
| ✓ | BbhBooleanExpressionsBe228False and not True and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT757 |
| ✓ | BbhBooleanExpressionsBe229True or False and not ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 250 |
Reasoning - BBH boolean expressions (set 8)
29 passed3 failed |
90.6% |
IN33 111 OUT37 636 | |
| ✓ | BbhBooleanExpressionsBe230not ( False ) and ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT930 |
| ✓ | BbhBooleanExpressionsBe231False and not ( ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT553 |
| ✓ | BbhBooleanExpressionsBe232not False or ( not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 080 |
| ✓ | BbhBooleanExpressionsBe233( ( not False ) and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT422 |
| ✓ | BbhBooleanExpressionsBe234True or False and not False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 446 |
| ✓ | BbhBooleanExpressionsBe235( ( not True and True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT827 |
| ✓ | BbhBooleanExpressionsBe236not ( False ) or not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT805 |
| ✓ | BbhBooleanExpressionsBe237not ( False or False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 063 |
| ✓ | BbhBooleanExpressionsBe238False and not ( not not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT937 |
| ✓ | BbhBooleanExpressionsBe239False and False and not ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT636 |
| ✓ | BbhBooleanExpressionsBe240False or ( not False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT681 |
| ✓ | BbhBooleanExpressionsBe241False or ( not not not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT540 |
| ✓ | BbhBooleanExpressionsBe242( not False ) or not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT587 |
| ✓ | BbhBooleanExpressionsBe243not False and False and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT704 |
| ✓ | BbhBooleanExpressionsBe244True or not False and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 350 |
| ✓ | BbhBooleanExpressionsBe245( not ( False ) ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT332 |
| ✓ | BbhBooleanExpressionsBe246True and not True or False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 664 |
| ✓ | BbhBooleanExpressionsBe247not ( not not not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT604 |
| ✓ | BbhBooleanExpressionsBe248True and ( not True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT419 |
| ✓ | BbhBooleanExpressionsBe249( not not not False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT805 |
| × | BbhCausalJudgementBe10judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("Yes")
Context: BbhCausalJudgementBe10: expected Yes, got [No] |
Failed | IN1 111 OUT1 826 |
| × | BbhCausalJudgementBe11judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("No")
Context: BbhCausalJudgementBe11: expected No, got [Yes] |
Failed | IN1 127 OUT2 075 |
| ✓ | BbhCausalJudgementBe12judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT2 719 |
| ✓ | BbhCausalJudgementBe13judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 166 OUT2 055 |
| × | BbhCausalJudgementBe14judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("Yes")
Context: BbhCausalJudgementBe14: expected Yes, got [No] |
Failed | IN1 145 OUT1 716 |
| ✓ | BbhCausalJudgementBe15judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 180 OUT1 601 |
| ✓ | BbhCausalJudgementBe16judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 168 OUT1 139 |
| ✓ | BbhCausalJudgementBe17judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT1 931 |
| ✓ | BbhCausalJudgementBe18judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 351 OUT2 118 |
| ✓ | BbhCausalJudgementBe19judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT1 127 |
| ✓ | BbhCausalJudgementBe8judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 128 |
| ✓ | BbhCausalJudgementBe9judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 816 |
Reasoning - BBH Dyck languages - close the brackets (set 1)
25 passed5 failed |
83.3% |
IN29 711 OUT106 814 | |
| ✓ | BbhDyckLanguagesBi13correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT729 |
| × | BbhDyckLanguagesBi14correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> )")
Context: BbhDyckLanguagesBi14: expected > ), got [] > )] |
Failed | IN1 007 OUT8 289 |
| ✓ | BbhDyckLanguagesBi15correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT2 765 |
| ✓ | BbhDyckLanguagesBi16correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT1 982 |
| ✓ | BbhDyckLanguagesBi17correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 315 |
| ✓ | BbhDyckLanguagesBi18correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT1 684 |
| ✓ | BbhDyckLanguagesBi19correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT2 597 |
| ✓ | BbhDyckLanguagesBi20correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 062 OUT11 490 |
| ✓ | BbhDyckLanguagesBi21correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 279 |
| × | BbhDyckLanguagesBi22correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} ] >")
Context: BbhDyckLanguagesBi22: expected } ] >, got [} }>] |
Failed | IN1 024 OUT9 460 |
| ✓ | BbhDyckLanguagesBi23correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT2 343 |
| ✓ | BbhDyckLanguagesBi24correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT2 277 |
| ✓ | BbhDyckLanguagesBi25correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT3 681 |
| ✓ | BbhDyckLanguagesBi26correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT1 230 |
| ✓ | BbhDyckLanguagesBi27correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT2 125 |
| ✓ | BbhDyckLanguagesBi28correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT1 167 |
| × | BbhDyckLanguagesBi29correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] ] ]")
Context: BbhDyckLanguagesBi29: expected ] ] ], got [}]]]] |
Failed | IN992 OUT3 461 |
| ✓ | BbhDyckLanguagesBi30correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT1 851 |
| ✓ | BbhDyckLanguagesBi31correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 011 OUT5 890 |
| ✓ | BbhDyckLanguagesBi32correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT4 589 |
| ✓ | BbhDyckLanguagesBi33correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 020 OUT5 236 |
| ✓ | BbhDyckLanguagesBi34correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT1 448 |
| × | BbhDyckLanguagesBi35correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("}")
Context: BbhDyckLanguagesBi35: expected }, got [{ { { } } }] |
Failed | IN976 OUT4 098 |
| ✓ | BbhDyckLanguagesBi36correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT3 853 |
| ✓ | BbhDyckLanguagesBi37correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 168 |
| ✓ | BbhDyckLanguagesBi38correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 709 |
| × | BbhDyckLanguagesBi39correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} ) )")
Context: BbhDyckLanguagesBi39: expected } ) ), got [} ) ) )] |
Failed | IN1 054 OUT10 473 |
| ✓ | BbhDyckLanguagesBi40correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT2 752 |
| ✓ | BbhDyckLanguagesBi41correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT1 820 |
| ✓ | BbhDyckLanguagesBi42correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT4 053 |
Reasoning - BBH Dyck languages - close the brackets (set 2)
29 passed2 failed1 broken |
90.6% |
IN31 926 OUT121 618 | |
| ✓ | BbhDyckLanguagesBj51correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT2 364 |
| ✓ | BbhDyckLanguagesBj52correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT2 413 |
| ✓ | BbhDyckLanguagesBj53correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT1 720 |
| ! | BbhDyckLanguagesBj54correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 064 OUT13 898 |
| ✓ | BbhDyckLanguagesBj55correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT3 269 |
| ✓ | BbhDyckLanguagesBj56correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 426 |
| ✓ | BbhDyckLanguagesBj57correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT825 |
| ✓ | BbhDyckLanguagesBj58correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 204 |
| ✓ | BbhDyckLanguagesBj59correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT896 |
| ✓ | BbhDyckLanguagesBj60correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT2 309 |
| × | BbhDyckLanguagesBj61correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("}")
Context: BbhDyckLanguagesBj61: expected }, got [{ < { ( { [ ] } ) } > { } ( [ { { { ( [ < > ] ) } } } ] ) { { ( { } ) } }] |
Failed | IN1 008 OUT3 931 |
| ✓ | BbhDyckLanguagesBj62correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT4 117 |
| ✓ | BbhDyckLanguagesBj63correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT1 441 |
| ✓ | BbhDyckLanguagesBj64correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT2 237 |
| ✓ | BbhDyckLanguagesBj65correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT1 635 |
| ✓ | BbhDyckLanguagesBj66correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 026 OUT8 912 |
| ✓ | BbhDyckLanguagesBj67correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT1 720 |
| ✓ | BbhDyckLanguagesBj68correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT1 440 |
| ✓ | BbhDyckLanguagesBj69correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT4 832 |
| ✓ | BbhDyckLanguagesBj70correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 002 OUT3 584 |
| × | BbhDyckLanguagesBj71correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") )")
Context: BbhDyckLanguagesBj71: expected ) ), got [])]])}{)>))] |
Failed | IN1 027 OUT9 538 |
| ✓ | BbhDyckLanguagesBj72correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT1 071 |
| ✓ | BbhDyckLanguagesBj73correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 035 OUT8 082 |
| ✓ | BbhDyckLanguagesBj74correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 070 OUT7 968 |
| ✓ | BbhDyckLanguagesBj75correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 028 OUT5 872 |
| ✓ | BbhDyckLanguagesBj76correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 503 |
| ✓ | BbhDyckLanguagesBj77correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 479 |
| ✓ | BbhDyckLanguagesBj78correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 020 OUT7 415 |
| ✓ | BbhDyckLanguagesBj79correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 006 OUT4 815 |
| ✓ | BbhDyckLanguagesBj80correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT3 001 |
| ✓ | BbhDyckLanguagesBj81correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT4 829 |
| ✓ | BbhDyckLanguagesBj82correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 872 |
Reasoning - BBH Dyck languages - close the brackets (set 3)
27 passed3 failed2 broken |
84.4% |
IN31 902 OUT121 465 | |
| ✓ | BbhDyckLanguagesBj100correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 578 |
| ✓ | BbhDyckLanguagesBj101correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 048 |
| ! | BbhDyckLanguagesBj102correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 045 OUT13 693 |
| ✓ | BbhDyckLanguagesBj103correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 899 |
| ✓ | BbhDyckLanguagesBj104correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 211 |
| ✓ | BbhDyckLanguagesBj105correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 288 |
| ✓ | BbhDyckLanguagesBj106correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 008 OUT7 339 |
| ✓ | BbhDyckLanguagesBj107correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 126 |
| ✓ | BbhDyckLanguagesBj108correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 719 |
| ✓ | BbhDyckLanguagesBj109correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 064 OUT9 464 |
| ✓ | BbhDyckLanguagesBj110correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 562 |
| ✓ | BbhDyckLanguagesBj111correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 010 OUT5 805 |
| × | BbhDyckLanguagesBj112correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ]")
Context: BbhDyckLanguagesBj112: expected > ], got [>)] |
Failed | IN1 063 OUT9 493 |
| ✓ | BbhDyckLanguagesBj113correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT694 |
| ✓ | BbhDyckLanguagesBj114correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT3 856 |
| ✓ | BbhDyckLanguagesBj83correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT1 745 |
| ✓ | BbhDyckLanguagesBj84correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT2 331 |
| ✓ | BbhDyckLanguagesBj85correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT2 076 |
| ✓ | BbhDyckLanguagesBj86correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT988 |
| ✓ | BbhDyckLanguagesBj87correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 029 OUT6 139 |
| × | BbhDyckLanguagesBj88correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} }")
Context: BbhDyckLanguagesBj88: expected } }, got [)>}}] |
Failed | IN1 057 OUT9 935 |
| ✓ | BbhDyckLanguagesBj89correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT1 210 |
| ✓ | BbhDyckLanguagesBj90correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT5 442 |
| ✓ | BbhDyckLanguagesBj91correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT2 181 |
| ✓ | BbhDyckLanguagesBj92correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 974 |
| ✓ | BbhDyckLanguagesBj93correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT2 636 |
| ✓ | BbhDyckLanguagesBj94correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 015 OUT4 828 |
| ! | BbhDyckLanguagesBj95correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 057 OUT9 713 |
| ✓ | BbhDyckLanguagesBj96correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT2 490 |
| ✓ | BbhDyckLanguagesBj97correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT1 997 |
| ✓ | BbhDyckLanguagesBj98correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT1 129 |
| × | BbhDyckLanguagesBj99correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") )")
Context: BbhDyckLanguagesBj99: expected ) ), got [)] |
Failed | IN993 OUT2 876 |
Reasoning - BBH Dyck languages - close the brackets (set 4)
23 passed7 failed2 broken |
71.9% |
IN31 845 OUT199 518 | |
| ✓ | BbhDyckLanguagesBj115correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 476 |
| × | BbhDyckLanguagesBj116correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> )")
Context: BbhDyckLanguagesBj116: expected > ), got [)>] |
Failed | IN1 003 OUT7 625 |
| ✓ | BbhDyckLanguagesBj117correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 005 OUT3 523 |
| ✓ | BbhDyckLanguagesBj118correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT5 812 |
| ✓ | BbhDyckLanguagesBj119correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 765 |
| ! | BbhDyckLanguagesBj120correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 017 OUT62 144 |
| ✓ | BbhDyckLanguagesBj121correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT3 078 |
| ! | BbhDyckLanguagesBj122correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 040 OUT19 507 |
| × | BbhDyckLanguagesBj123correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ]")
Context: BbhDyckLanguagesBj123: expected > ], got [[ < [ ] > ]] |
Failed | IN975 OUT1 503 |
| ✓ | BbhDyckLanguagesBj124correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 002 OUT4 409 |
| × | BbhDyckLanguagesBj125correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ]")
Context: BbhDyckLanguagesBj125: expected > ], got [] > ]] |
Failed | IN997 OUT2 605 |
| ✓ | BbhDyckLanguagesBj126correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT2 208 |
| ✓ | BbhDyckLanguagesBj127correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 015 OUT7 531 |
| ✓ | BbhDyckLanguagesBj128correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT1 364 |
| ✓ | BbhDyckLanguagesBj129correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 367 |
| ✓ | BbhDyckLanguagesBj130correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT4 588 |
| ✓ | BbhDyckLanguagesBj131correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 007 OUT3 344 |
| ✓ | BbhDyckLanguagesBj132correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT2 462 |
| ✓ | BbhDyckLanguagesBj133correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT5 605 |
| ✓ | BbhDyckLanguagesBj134correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 432 |
| ✓ | BbhDyckLanguagesBj135correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 022 OUT13 154 |
| ✓ | BbhDyckLanguagesBj136correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT3 924 |
| ✓ | BbhDyckLanguagesBj137correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT2 329 |
| ✓ | BbhDyckLanguagesBj138correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT3 548 |
| × | BbhDyckLanguagesBj139correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBj139: expected ), got [] |
Failed | IN1 048 OUT8 301 |
| ✓ | BbhDyckLanguagesBj140correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 008 OUT5 576 |
| × | BbhDyckLanguagesBj141correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") )")
Context: BbhDyckLanguagesBj141: expected ) ), got [( ) ( () )] |
Failed | IN975 OUT2 362 |
| × | BbhDyckLanguagesBj142correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ) }")
Context: BbhDyckLanguagesBj142: expected > ) }, got [)>}] |
Failed | IN982 OUT1 594 |
| ✓ | BbhDyckLanguagesBj143correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT2 587 |
| ✓ | BbhDyckLanguagesBj144correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT3 984 |
| × | BbhDyckLanguagesBj145correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") > ]")
Context: BbhDyckLanguagesBj145: expected ) > ], got [)]>]] |
Failed | IN1 046 OUT6 585 |
| ✓ | BbhDyckLanguagesBj146correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT2 226 |
Reasoning - BBH Dyck languages - close the brackets (set 5)
29 passed2 failed1 broken |
90.6% |
IN31 748 OUT114 389 | |
| ✓ | BbhDyckLanguagesBj147correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT2 905 |
| ✓ | BbhDyckLanguagesBj148correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 016 OUT6 460 |
| ✓ | BbhDyckLanguagesBj149correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 569 |
| ✓ | BbhDyckLanguagesBj150correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT1 792 |
| ✓ | BbhDyckLanguagesBj151correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT577 |
| ✓ | BbhDyckLanguagesBj152correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT3 968 |
| ✓ | BbhDyckLanguagesBj153correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 007 OUT2 921 |
| ✓ | BbhDyckLanguagesBj154correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 006 OUT3 860 |
| ✓ | BbhDyckLanguagesBj155correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 009 OUT6 832 |
| ✓ | BbhDyckLanguagesBj156correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 628 |
| ✓ | BbhDyckLanguagesBj157correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT2 570 |
| ✓ | BbhDyckLanguagesBj158correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT1 527 |
| ✓ | BbhDyckLanguagesBj159correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT2 047 |
| ✓ | BbhDyckLanguagesBj160correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 018 OUT4 676 |
| ✓ | BbhDyckLanguagesBj161correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT923 |
| ✓ | BbhDyckLanguagesBj162correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT2 960 |
| ✓ | BbhDyckLanguagesBj163correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT2 067 |
| ✓ | BbhDyckLanguagesBj164correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT2 853 |
| ✓ | BbhDyckLanguagesBj165correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT2 452 |
| ✓ | BbhDyckLanguagesBj166correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 704 |
| ✓ | BbhDyckLanguagesBj167correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 006 OUT5 033 |
| ✓ | BbhDyckLanguagesBj168correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT971 |
| × | BbhDyckLanguagesBj169correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] ) }")
Context: BbhDyckLanguagesBj169: expected ] ) }, got [])}}] |
Failed | IN986 OUT1 962 |
| ✓ | BbhDyckLanguagesBj170correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 021 OUT4 350 |
| ✓ | BbhDyckLanguagesBj171correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT4 777 |
| ✓ | BbhDyckLanguagesBj172correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT3 293 |
| × | BbhDyckLanguagesBj173correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> > >")
Context: BbhDyckLanguagesBj173: expected > > >, got [> ] > >] |
Failed | IN1 000 OUT9 309 |
| ! | BbhDyckLanguagesBj174correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 062 OUT18 587 |
| ✓ | BbhDyckLanguagesBj175correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 002 OUT2 988 |
| ✓ | BbhDyckLanguagesBj176correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 337 |
| ✓ | BbhDyckLanguagesBj177correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT2 017 |
| ✓ | BbhDyckLanguagesBj178correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT3 474 |
Reasoning - BBH Dyck languages - close the brackets (set 6)
22 passed6 failed2 broken |
73.3% |
IN29 880 OUT189 775 | |
| ✓ | BbhDyckLanguagesBj179correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 750 |
| ✓ | BbhDyckLanguagesBj180correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT2 753 |
| ✓ | BbhDyckLanguagesBj181correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 363 |
| ✓ | BbhDyckLanguagesBj182correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 320 |
| ✓ | BbhDyckLanguagesBj183correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT2 062 |
| × | BbhDyckLanguagesBj184correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(">")
Context: BbhDyckLanguagesBj184: expected >, got [> ] ) }] |
Failed | IN1 066 OUT11 906 |
| ✓ | BbhDyckLanguagesBj185correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT2 182 |
| × | BbhDyckLanguagesBj186correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ] ]")
Context: BbhDyckLanguagesBj186: expected ) ] ], got [} ) ] ] ]] |
Failed | IN1 034 OUT19 329 |
| ✓ | BbhDyckLanguagesBj187correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT587 |
| × | BbhDyckLanguagesBj188correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} )")
Context: BbhDyckLanguagesBj188: expected } ), got [( { } )] |
Failed | IN973 OUT745 |
| × | BbhDyckLanguagesBj189correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ]")
Context: BbhDyckLanguagesBj189: expected ) ], got [)] |
Failed | IN1 049 OUT11 060 |
| ✓ | BbhDyckLanguagesBj190correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 756 |
| × | BbhDyckLanguagesBj191correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") >")
Context: BbhDyckLanguagesBj191: expected ) >, got [< ( ) ( { { [ ] } } ) >] |
Failed | IN981 OUT2 538 |
| ✓ | BbhDyckLanguagesBj192correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 337 |
| ✓ | BbhDyckLanguagesBj193correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT1 137 |
| × | BbhDyckLanguagesBj194correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("]")
Context: BbhDyckLanguagesBj194: expected ], got [} ))]]>[] |
Failed | IN1 002 OUT5 393 |
| ! | BbhDyckLanguagesBj195correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 035 OUT62 144 |
| ✓ | BbhDyckLanguagesBj196correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT813 |
| ✓ | BbhDyckLanguagesBj197correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT1 211 |
| ✓ | BbhDyckLanguagesBj198correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 044 OUT7 996 |
| ✓ | BbhDyckLanguagesBj199correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 019 OUT4 022 |
| ! | BbhDyckLanguagesBj200correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Skipped: AI output broken (model loop aborted) |
Broken | IN1 064 OUT22 395 |
| ✓ | BbhDyckLanguagesBj201correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT4 137 |
| ✓ | BbhDyckLanguagesBj202correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 659 |
| ✓ | BbhDyckLanguagesBj203correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 623 |
| ✓ | BbhDyckLanguagesBj204correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT4 836 |
| ✓ | BbhDyckLanguagesBj205correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT3 398 |
| ✓ | BbhDyckLanguagesBj206correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT2 056 |
| ✓ | BbhDyckLanguagesBj207correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT4 528 |
| ✓ | BbhDyckLanguagesBj208correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 739 |
Reasoning - BBH Dyck languages - close the brackets (set 7)
18 passed14 failed |
56.3% |
IN31 962 OUT120 201 | |
| × | BbhDyckLanguagesBk209correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} } )")
Context: BbhDyckLanguagesBk209: expected } } ), got [}] |
Failed | IN1 026 OUT7 672 |
| ✓ | BbhDyckLanguagesBk210correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT1 856 |
| × | BbhDyckLanguagesBk211correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] } )")
Context: BbhDyckLanguagesBk211: expected ] } ), got [] }] |
Failed | IN976 OUT3 622 |
| × | BbhDyckLanguagesBk212correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} )")
Context: BbhDyckLanguagesBk212: expected } ), got []}] |
Failed | IN1 067 OUT9 324 |
| × | BbhDyckLanguagesBk213correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") >")
Context: BbhDyckLanguagesBk213: expected ) >, got [>] |
Failed | IN987 OUT2 874 |
| ✓ | BbhDyckLanguagesBk214correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT3 831 |
| × | BbhDyckLanguagesBk215correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") > }")
Context: BbhDyckLanguagesBk215: expected ) > }, got [> }] |
Failed | IN1 028 OUT6 610 |
| ✓ | BbhDyckLanguagesBk216correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT1 291 |
| × | BbhDyckLanguagesBk217correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ] ]")
Context: BbhDyckLanguagesBk217: expected > ] ], got [] ] >] |
Failed | IN974 OUT1 864 |
| × | BbhDyckLanguagesBk218correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBk218: expected ), got [] |
Failed | IN974 OUT891 |
| ✓ | BbhDyckLanguagesBk219correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT1 895 |
| ✓ | BbhDyckLanguagesBk220correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT1 226 |
| × | BbhDyckLanguagesBk221correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ] ]")
Context: BbhDyckLanguagesBk221: expected ) ] ], got []]] |
Failed | IN1 002 OUT3 291 |
| ✓ | BbhDyckLanguagesBk222correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT1 712 |
| ✓ | BbhDyckLanguagesBk223correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT2 657 |
| ✓ | BbhDyckLanguagesBk224correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT1 114 |
| × | BbhDyckLanguagesBk225correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} ] }")
Context: BbhDyckLanguagesBk225: expected } ] }, got [}]}}]}`] |
Failed | IN1 064 OUT4 982 |
| ✓ | BbhDyckLanguagesBk226correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 007 OUT5 438 |
| × | BbhDyckLanguagesBk227correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} } )")
Context: BbhDyckLanguagesBk227: expected } } ), got [} }] |
Failed | IN1 068 OUT7 502 |
| × | BbhDyckLanguagesBk228correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") >")
Context: BbhDyckLanguagesBk228: expected ) >, got [>] |
Failed | IN973 OUT964 |
| ✓ | BbhDyckLanguagesBk229correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT1 949 |
| ✓ | BbhDyckLanguagesBk230correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT3 897 |
| ✓ | BbhDyckLanguagesBk231correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT3 520 |
| × | BbhDyckLanguagesBk232correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] >")
Context: BbhDyckLanguagesBk232: expected ] >, got [}]>] |
Failed | IN991 OUT1 354 |
| ✓ | BbhDyckLanguagesBk233correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT2 462 |
| ✓ | BbhDyckLanguagesBk234correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 965 |
| × | BbhDyckLanguagesBk235correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(">")
Context: BbhDyckLanguagesBk235: expected >, got []>}] |
Failed | IN1 032 OUT14 993 |
| × | BbhDyckLanguagesBk236correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") } >")
Context: BbhDyckLanguagesBk236: expected ) } >, got [}>] |
Failed | IN988 OUT2 799 |
| ✓ | BbhDyckLanguagesBk237correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 068 OUT10 788 |
| ✓ | BbhDyckLanguagesBk238correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 191 |
| ✓ | BbhDyckLanguagesBk239correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 591 |
| ✓ | BbhDyckLanguagesBk240correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT3 076 |
Reasoning - BBH formal fallacies (set 1)
26 passed6 failed |
81.3% |
IN33 742 OUT77 648 | |
| × | BbhDyckLanguagesBk241correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBk241: expected ), got [] |
Failed | IN980 OUT4 604 |
| × | BbhDyckLanguagesBk242correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") }")
Context: BbhDyckLanguagesBk242: expected ) }, got [}] |
Failed | IN981 OUT2 285 |
| × | BbhDyckLanguagesBk243correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} )")
Context: BbhDyckLanguagesBk243: expected } ), got [}] |
Failed | IN997 OUT2 747 |
| × | BbhDyckLanguagesBk244correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ) }")
Context: BbhDyckLanguagesBk244: expected > ) }, got [> }] |
Failed | IN974 OUT3 020 |
| × | BbhDyckLanguagesBk245correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} ) }")
Context: BbhDyckLanguagesBk245: expected } ) }, got [} }] |
Failed | IN996 OUT4 224 |
| ✓ | BbhDyckLanguagesBk246correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 017 OUT5 086 |
| × | BbhDyckLanguagesBk247correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ) )")
Context: BbhDyckLanguagesBk247: expected > ) ), got [>] |
Failed | IN990 OUT1 025 |
| ✓ | BbhDyckLanguagesBk248correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 930 |
| ✓ | BbhDyckLanguagesBk249correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 556 |
| ✓ | BbhFormalFallaciesBk10decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 313 |
| ✓ | BbhFormalFallaciesBk11decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 368 |
| ✓ | BbhFormalFallaciesBk12decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT1 452 |
| ✓ | BbhFormalFallaciesBk13decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 351 |
| ✓ | BbhFormalFallaciesBk14decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT3 318 |
| ✓ | BbhFormalFallaciesBk15decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT3 660 |
| ✓ | BbhFormalFallaciesBk16decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 805 |
| ✓ | BbhFormalFallaciesBk17decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 504 |
| ✓ | BbhFormalFallaciesBk18decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT3 155 |
| ✓ | BbhFormalFallaciesBk19decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT1 424 |
| ✓ | BbhFormalFallaciesBk20decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT4 034 |
| ✓ | BbhFormalFallaciesBk21decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT2 482 |
| ✓ | BbhFormalFallaciesBk22decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 742 |
| ✓ | BbhFormalFallaciesBk23decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 914 |
| ✓ | BbhFormalFallaciesBk24decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT3 520 |
| ✓ | BbhFormalFallaciesBk25decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 633 |
| ✓ | BbhFormalFallaciesBk26decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT817 |
| ✓ | BbhFormalFallaciesBk27decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT2 692 |
| ✓ | BbhFormalFallaciesBk28decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT2 803 |
| ✓ | BbhFormalFallaciesBk29decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 631 |
| ✓ | BbhFormalFallaciesBk30decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT5 094 |
| ✓ | BbhFormalFallaciesBk8decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT1 312 |
| ✓ | BbhFormalFallaciesBk9decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 147 |
Reasoning - BBH formal fallacies (set 2)
31 passed1 failed |
96.9% |
IN34 351 OUT66 541 | |
| ✓ | BbhFormalFallaciesBk31decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT2 092 |
| ✓ | BbhFormalFallaciesBk32decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT2 008 |
| ✓ | BbhFormalFallaciesBk33decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT2 060 |
| × | BbhFormalFallaciesBk34decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("valid")
Context: BbhFormalFallaciesBk34: expected valid, got [invalid]
Values:
got = "invalid" |
Failed | IN1 105 OUT3 510 |
| ✓ | BbhFormalFallaciesBk35decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 902 |
| ✓ | BbhFormalFallaciesBk36decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT3 590 |
| ✓ | BbhFormalFallaciesBk37decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT4 334 |
| ✓ | BbhFormalFallaciesBk38decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 647 |
| ✓ | BbhFormalFallaciesBk39decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT2 532 |
| ✓ | BbhFormalFallaciesBk40decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT2 484 |
| ✓ | BbhFormalFallaciesBk41decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT993 |
| ✓ | BbhFormalFallaciesBk42decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 675 |
| ✓ | BbhFormalFallaciesBk43decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT1 643 |
| ✓ | BbhFormalFallaciesBk44decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 574 |
| ✓ | BbhFormalFallaciesBk45decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT1 413 |
| ✓ | BbhFormalFallaciesBk46decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT2 613 |
| ✓ | BbhFormalFallaciesBk47decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT1 614 |
| ✓ | BbhFormalFallaciesBk48decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT1 308 |
| ✓ | BbhFormalFallaciesBk49decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT2 388 |
| ✓ | BbhFormalFallaciesBk50decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT1 920 |
| ✓ | BbhFormalFallaciesBk51decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT1 958 |
| ✓ | BbhFormalFallaciesBk52decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT1 038 |
| ✓ | BbhFormalFallaciesBk53decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 285 |
| ✓ | BbhFormalFallaciesBk54decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT1 392 |
| ✓ | BbhFormalFallaciesBk55decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 259 |
| ✓ | BbhFormalFallaciesBk56decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT1 431 |
| ✓ | BbhFormalFallaciesBk57decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 275 |
| ✓ | BbhFormalFallaciesBk58decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT3 740 |
| ✓ | BbhFormalFallaciesBk59decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT2 744 |
| ✓ | BbhFormalFallaciesBk60decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT2 161 |
| ✓ | BbhFormalFallaciesBk61decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT1 795 |
| ✓ | BbhFormalFallaciesBk62decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT2 163 |
Reasoning - BBH formal fallacies (set 3)
32 passed |
100.0% |
IN34 560 OUT73 067 | |
| ✓ | BbhFormalFallaciesBk63decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT2 517 |
| ✓ | BbhFormalFallaciesBk64decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT1 610 |
| ✓ | BbhFormalFallaciesBk65decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT2 633 |
| ✓ | BbhFormalFallaciesBk66decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 411 |
| ✓ | BbhFormalFallaciesBk67decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT2 104 |
| ✓ | BbhFormalFallaciesBk68decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT3 081 |
| ✓ | BbhFormalFallaciesBk69decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT2 713 |
| ✓ | BbhFormalFallaciesBk70decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT1 789 |
| ✓ | BbhFormalFallaciesBk71decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 031 |
| ✓ | BbhFormalFallaciesBk72decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 530 |
| ✓ | BbhFormalFallaciesBk73decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 842 |
| ✓ | BbhFormalFallaciesBk74decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT2 661 |
| ✓ | BbhFormalFallaciesBk75decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT2 748 |
| ✓ | BbhFormalFallaciesBk76decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT3 277 |
| ✓ | BbhFormalFallaciesBk77decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT2 514 |
| ✓ | BbhFormalFallaciesBk78decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 822 |
| ✓ | BbhFormalFallaciesBk79decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT2 402 |
| ✓ | BbhFormalFallaciesBk80decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT3 253 |
| ✓ | BbhFormalFallaciesBk81decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT3 742 |
| ✓ | BbhFormalFallaciesBk82decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT891 |
| ✓ | BbhFormalFallaciesBk83decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT2 483 |
| ✓ | BbhFormalFallaciesBk84decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 159 |
| ✓ | BbhFormalFallaciesBk85decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 641 |
| ✓ | BbhFormalFallaciesBk86decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT3 043 |
| ✓ | BbhFormalFallaciesBk87decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT2 744 |
| ✓ | BbhFormalFallaciesBk88decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT3 777 |
| ✓ | BbhFormalFallaciesBk89decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT1 418 |
| ✓ | BbhFormalFallaciesBk90decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT3 000 |
| ✓ | BbhFormalFallaciesBk91decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT2 273 |
| ✓ | BbhFormalFallaciesBk92decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 857 |
| ✓ | BbhFormalFallaciesBk93decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 818 |
| ✓ | BbhFormalFallaciesBk94decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 283 |
Reasoning - BBH formal fallacies (set 4)
30 passed |
100.0% |
IN32 128 OUT58 153 | |
| ✓ | BbhFormalFallaciesBk100decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT1 804 |
| ✓ | BbhFormalFallaciesBk101decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT2 755 |
| ✓ | BbhFormalFallaciesBk102decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT1 206 |
| ✓ | BbhFormalFallaciesBk103decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 589 |
| ✓ | BbhFormalFallaciesBk104decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT2 964 |
| ✓ | BbhFormalFallaciesBk105decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT3 235 |
| ✓ | BbhFormalFallaciesBk106decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT1 798 |
| ✓ | BbhFormalFallaciesBk107decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT2 133 |
| ✓ | BbhFormalFallaciesBk108decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT2 861 |
| ✓ | BbhFormalFallaciesBk109decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 575 |
| ✓ | BbhFormalFallaciesBk110decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 783 |
| ✓ | BbhFormalFallaciesBk111decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT2 045 |
| ✓ | BbhFormalFallaciesBk112decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT1 102 |
| ✓ | BbhFormalFallaciesBk113decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT2 363 |
| ✓ | BbhFormalFallaciesBk114decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 418 |
| ✓ | BbhFormalFallaciesBk115decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 405 |
| ✓ | BbhFormalFallaciesBk116decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT1 049 |
| ✓ | BbhFormalFallaciesBk117decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 162 |
| ✓ | BbhFormalFallaciesBk118decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 251 |
| ✓ | BbhFormalFallaciesBk119decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 404 |
| ✓ | BbhFormalFallaciesBk120decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT994 |
| ✓ | BbhFormalFallaciesBk121decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 345 |
| ✓ | BbhFormalFallaciesBk122decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT1 990 |
| ✓ | BbhFormalFallaciesBk123decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 671 |
| ✓ | BbhFormalFallaciesBk124decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT1 225 |
| ✓ | BbhFormalFallaciesBk95decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT3 315 |
| ✓ | BbhFormalFallaciesBk96decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT2 828 |
| ✓ | BbhFormalFallaciesBk97decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT3 637 |
| ✓ | BbhFormalFallaciesBk98decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 361 |
| ✓ | BbhFormalFallaciesBk99decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT2 885 |
Reasoning - BBH formal fallacies (set 5)
32 passed |
100.0% |
IN34 430 OUT68 846 | |
| ✓ | BbhFormalFallacies125BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT2 180 |
| ✓ | BbhFormalFallacies126BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT1 641 |
| ✓ | BbhFormalFallacies127BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT1 058 |
| ✓ | BbhFormalFallacies128BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT1 062 |
| ✓ | BbhFormalFallacies129BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT979 |
| ✓ | BbhFormalFallacies130BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT1 561 |
| ✓ | BbhFormalFallacies131BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 899 |
| ✓ | BbhFormalFallacies132BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT2 312 |
| ✓ | BbhFormalFallacies133BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT839 |
| ✓ | BbhFormalFallacies134BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 041 OUT882 |
| ✓ | BbhFormalFallacies135BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT2 436 |
| ✓ | BbhFormalFallacies136BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT2 800 |
| ✓ | BbhFormalFallacies137BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 635 |
| ✓ | BbhFormalFallacies138BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 201 |
| ✓ | BbhFormalFallacies139BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT2 060 |
| ✓ | BbhFormalFallacies140BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 826 |
| ✓ | BbhFormalFallacies141BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT1 929 |
| ✓ | BbhFormalFallacies142BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT2 569 |
| ✓ | BbhFormalFallacies143BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 572 |
| ✓ | BbhFormalFallacies144BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT2 336 |
| ✓ | BbhFormalFallacies145BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT2 190 |
| ✓ | BbhFormalFallacies146BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 388 |
| ✓ | BbhFormalFallacies147BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT2 156 |
| ✓ | BbhFormalFallacies148BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT3 658 |
| ✓ | BbhFormalFallacies149BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT3 103 |
| ✓ | BbhFormalFallacies150BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT4 083 |
| ✓ | BbhFormalFallacies151BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 644 |
| ✓ | BbhFormalFallacies152BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT2 043 |
| ✓ | BbhFormalFallacies153BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT2 616 |
| ✓ | BbhFormalFallacies154BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT1 908 |
| ✓ | BbhFormalFallacies155BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 481 |
| ✓ | BbhFormalFallacies156BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT3 799 |
Reasoning - BBH formal fallacies (set 6)
32 passed |
100.0% |
IN34 425 OUT65 570 | |
| ✓ | BbhFormalFallacies157BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 687 |
| ✓ | BbhFormalFallacies158BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 736 |
| ✓ | BbhFormalFallacies159BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT2 164 |
| ✓ | BbhFormalFallacies160BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT2 918 |
| ✓ | BbhFormalFallacies161BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 413 |
| ✓ | BbhFormalFallacies162BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 667 |
| ✓ | BbhFormalFallacies163BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT2 778 |
| ✓ | BbhFormalFallacies164BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT3 762 |
| ✓ | BbhFormalFallacies165BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT2 998 |
| ✓ | BbhFormalFallacies166BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 938 |
| ✓ | BbhFormalFallacies167BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 566 |
| ✓ | BbhFormalFallacies168BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 041 OUT1 454 |
| ✓ | BbhFormalFallacies169BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT2 686 |
| ✓ | BbhFormalFallacies170BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT1 716 |
| ✓ | BbhFormalFallacies171BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT2 765 |
| ✓ | BbhFormalFallacies172BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT3 801 |
| ✓ | BbhFormalFallacies173BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 180 |
| ✓ | BbhFormalFallacies174BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT1 568 |
| ✓ | BbhFormalFallacies175BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT1 539 |
| ✓ | BbhFormalFallacies176BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 554 |
| ✓ | BbhFormalFallacies177BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT2 686 |
| ✓ | BbhFormalFallacies178BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT3 155 |
| ✓ | BbhFormalFallacies179BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 747 |
| ✓ | BbhFormalFallacies180BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT1 882 |
| ✓ | BbhFormalFallacies181BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 514 |
| ✓ | BbhFormalFallacies182BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT2 018 |
| ✓ | BbhFormalFallacies183BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 182 |
| ✓ | BbhFormalFallacies184BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT1 597 |
| ✓ | BbhFormalFallacies185BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 036 OUT1 002 |
| ✓ | BbhFormalFallacies186BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 080 |
| ✓ | BbhFormalFallacies187BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 123 OUT2 294 |
| ✓ | BbhFormalFallacies188BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT2 523 |
Reasoning - BBH formal fallacies (set 7)
32 passed |
100.0% |
IN34 299 OUT69 949 | |
| ✓ | BbhFormalFallacies189BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT927 |
| ✓ | BbhFormalFallacies190BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT2 282 |
| ✓ | BbhFormalFallacies191BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT1 761 |
| ✓ | BbhFormalFallacies192BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 790 |
| ✓ | BbhFormalFallacies193BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT1 755 |
| ✓ | BbhFormalFallacies194BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT936 |
| ✓ | BbhFormalFallacies195BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT1 605 |
| ✓ | BbhFormalFallacies196BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT2 969 |
| ✓ | BbhFormalFallacies197BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 769 |
| ✓ | BbhFormalFallacies198BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT2 140 |
| ✓ | BbhFormalFallacies199BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT4 844 |
| ✓ | BbhFormalFallacies200BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT2 178 |
| ✓ | BbhFormalFallacies201BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT2 665 |
| ✓ | BbhFormalFallacies202BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT4 903 |
| ✓ | BbhFormalFallacies203BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT1 732 |
| ✓ | BbhFormalFallacies204BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT872 |
| ✓ | BbhFormalFallacies205BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT2 739 |
| ✓ | BbhFormalFallacies206BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT2 777 |
| ✓ | BbhFormalFallacies207BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 486 |
| ✓ | BbhFormalFallacies208BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT2 399 |
| ✓ | BbhFormalFallacies209BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT1 836 |
| ✓ | BbhFormalFallacies210BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT907 |
| ✓ | BbhFormalFallacies211BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT2 632 |
| ✓ | BbhFormalFallacies212BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 452 |
| ✓ | BbhFormalFallacies213BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT2 758 |
| ✓ | BbhFormalFallacies214BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT2 037 |
| ✓ | BbhFormalFallacies215BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT2 796 |
| ✓ | BbhFormalFallacies216BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT2 890 |
| ✓ | BbhFormalFallacies217BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT2 622 |
| ✓ | BbhFormalFallacies218BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 221 |
| ✓ | BbhFormalFallacies219BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 362 |
| ✓ | BbhFormalFallacies220BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT1 907 |
Reasoning - BBH formal fallacies (set 8)
29 passed |
100.0% |
IN31 182 OUT69 084 | |
| ✓ | BbhFormalFallacies221BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT2 217 |
| ✓ | BbhFormalFallacies222BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 102 |
| ✓ | BbhFormalFallacies223BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT2 845 |
| ✓ | BbhFormalFallacies224BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT2 520 |
| ✓ | BbhFormalFallacies225BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT1 060 |
| ✓ | BbhFormalFallacies226BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT3 372 |
| ✓ | BbhFormalFallacies227BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 978 |
| ✓ | BbhFormalFallacies228BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT3 512 |
| ✓ | BbhFormalFallacies229BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 574 |
| ✓ | BbhFormalFallacies230BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT3 547 |
| ✓ | BbhFormalFallacies231BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT1 766 |
| ✓ | BbhFormalFallacies232BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT2 164 |
| ✓ | BbhFormalFallacies233BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT2 364 |
| ✓ | BbhFormalFallacies234BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT4 333 |
| ✓ | BbhFormalFallacies235BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT1 342 |
| ✓ | BbhFormalFallacies236BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 956 |
| ✓ | BbhFormalFallacies237BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT1 445 |
| ✓ | BbhFormalFallacies238BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 475 |
| ✓ | BbhFormalFallacies239BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT4 442 |
| ✓ | BbhFormalFallacies240BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT2 806 |
| ✓ | BbhFormalFallacies241BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT1 445 |
| ✓ | BbhFormalFallacies242BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT3 416 |
| ✓ | BbhFormalFallacies243BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT2 161 |
| ✓ | BbhFormalFallacies244BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT1 240 |
| ✓ | BbhFormalFallacies245BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT1 070 |
| ✓ | BbhFormalFallacies246BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT3 604 |
| ✓ | BbhFormalFallacies247BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT1 472 |
| ✓ | BbhFormalFallacies248BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT1 903 |
| ✓ | BbhFormalFallacies249BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 953 |
Reasoning - BBH geometric shapes (SVG paths) (set 1)
26 passed7 failed |
78.8% |
IN36 604 OUT74 455 | |
| ✓ | BbhGeometricShapes10BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT2 251 |
| ✓ | BbhGeometricShapes11BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT2 268 |
| × | BbhGeometricShapes12BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes12BL: expected (K), got [(A)] |
Failed | IN1 101 OUT2 188 |
| × | BbhGeometricShapes13BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes13BL: expected (K), got [(A)] |
Failed | IN1 097 OUT1 794 |
| × | BbhGeometricShapes14BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes14BL: expected (K), got [(H)] |
Failed | IN1 094 OUT3 480 |
| ✓ | BbhGeometricShapes15BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT2 822 |
| ✓ | BbhGeometricShapes16BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 049 |
| ✓ | BbhGeometricShapes17BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 194 OUT2 511 |
| × | BbhGeometricShapes18BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes18BL: expected (K), got [(H)] |
Failed | IN1 110 OUT3 329 |
| ✓ | BbhGeometricShapes19BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT4 113 |
| ✓ | BbhGeometricShapes20BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 526 |
| ✓ | BbhGeometricShapes21BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 168 OUT1 835 |
| ✓ | BbhGeometricShapes22BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 940 |
| ✓ | BbhGeometricShapes23BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 164 OUT2 033 |
| ✓ | BbhGeometricShapes24BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 168 OUT2 694 |
| ✓ | BbhGeometricShapes25BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT3 049 |
| ✓ | BbhGeometricShapes26BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 084 |
| ✓ | BbhGeometricShapes27BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 476 |
| ✓ | BbhGeometricShapes28BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 822 |
| × | BbhGeometricShapes29BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes29BL: expected (K), got [(H)] |
Failed | IN1 082 OUT1 414 |
| ✓ | BbhGeometricShapes30BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 704 |
| ✓ | BbhGeometricShapes31BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 566 |
| ✓ | BbhGeometricShapes32BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT2 680 |
| × | BbhGeometricShapes33BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes33BL: expected (K), got [(A)] |
Failed | IN1 101 OUT2 241 |
| ✓ | BbhGeometricShapes34BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT2 962 |
| ✓ | BbhGeometricShapes35BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 819 |
| ✓ | BbhGeometricShapes36BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT2 800 |
| ✓ | BbhGeometricShapes37BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 314 |
| ✓ | BbhGeometricShapes38BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 316 |
| ✓ | BbhGeometricShapes39BLThis SVG path element <path d="M 2.27,59.23 L 52.45,67.18"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT564 |
| ✓ | BbhGeometricShapes40BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 692 |
| × | BbhGeometricShapes8BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes8BL: expected (K), got [(H)] |
Failed | IN1 082 OUT4 334 |
| ✓ | BbhGeometricShapes9BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 785 |
Reasoning - BBH geometric shapes (SVG paths) (set 2)
24 passed8 failed |
75.0% |
IN35 033 OUT63 733 | |
| ✓ | BbhGeometricShapes41Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT3 383 |
| ✓ | BbhGeometricShapes42Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 428 |
| ✓ | BbhGeometricShapes43BmThis SVG path element <path d="M 98.96,92.35 L 44.27,28.27"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT629 |
| ✓ | BbhGeometricShapes44Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT944 |
| × | BbhGeometricShapes45Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes45Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 097 OUT1 889 |
| × | BbhGeometricShapes46Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes46Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT1 704 |
| ✓ | BbhGeometricShapes47Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 236 |
| ✓ | BbhGeometricShapes48Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 682 |
| ✓ | BbhGeometricShapes49Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 002 |
| ✓ | BbhGeometricShapes50Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 165 OUT1 826 |
| ✓ | BbhGeometricShapes51Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 101 |
| ✓ | BbhGeometricShapes52Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 564 |
| ✓ | BbhGeometricShapes53Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 774 |
| × | BbhGeometricShapes54Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhGeometricShapes54Bm: expected (B), got [c]
Values:
got = "c" |
Failed | IN1 116 OUT1 923 |
| ✓ | BbhGeometricShapes55BmThis SVG path element <path d="M 32.95,20.92 L 31.65,35.07"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT487 |
| ✓ | BbhGeometricShapes56Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 960 |
| ✓ | BbhGeometricShapes57Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 528 |
| ✓ | BbhGeometricShapes58Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT3 981 |
| × | BbhGeometricShapes59Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes59Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 095 OUT2 216 |
| ✓ | BbhGeometricShapes60Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 987 |
| ✓ | BbhGeometricShapes61Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 293 |
| ✓ | BbhGeometricShapes62Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT2 212 |
| ✓ | BbhGeometricShapes63Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 557 |
| ✓ | BbhGeometricShapes64Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 181 |
| × | BbhGeometricShapes65Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes65Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT1 581 |
| × | BbhGeometricShapes66Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes66Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 095 OUT4 443 |
| ✓ | BbhGeometricShapes67Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 482 |
| ✓ | BbhGeometricShapes68BmThis SVG path element <path d="M 11.79,16.93 L 56.17,80.16"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT1 360 |
| ✓ | BbhGeometricShapes69Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 693 |
| ✓ | BbhGeometricShapes70Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT1 292 |
| × | BbhGeometricShapes71Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes71Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT1 842 |
| × | BbhGeometricShapes72Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes72Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 084 OUT1 553 |
Reasoning - BBH geometric shapes (SVG paths) (set 3)
25 passed7 failed |
78.1% |
IN34 974 OUT63 948 | |
| ✓ | BbhGeometricShapes100Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 513 |
| ✓ | BbhGeometricShapes101Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 327 |
| ✓ | BbhGeometricShapes102Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT2 269 |
| ✓ | BbhGeometricShapes103Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT2 220 |
| × | BbhGeometricShapes104Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes104Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 096 OUT2 301 |
| ✓ | BbhGeometricShapes73BmThis SVG path element <path d="M 31.97,19.98 L 38.55,43.43"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT918 |
| ✓ | BbhGeometricShapes74BmThis SVG path element <path d="M 81.33,89.86 L 65.20,41.90"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT875 |
| ✓ | BbhGeometricShapes75BmThis SVG path element <path d="M 27.49,66.87 L 19.65,56.41"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT501 |
| × | BbhGeometricShapes76Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(I)")
Context: BbhGeometricShapes76Bm: expected (I), got [j]
Values:
got = "j" |
Failed | IN1 086 OUT2 172 |
| ✓ | BbhGeometricShapes77Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT2 410 |
| ✓ | BbhGeometricShapes78Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT3 678 |
| ✓ | BbhGeometricShapes79BmThis SVG path element <path d="M 14.31,45.36 L 17.22,37.11"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT1 032 |
| × | BbhGeometricShapes80Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes80Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 095 OUT3 654 |
| ✓ | BbhGeometricShapes81Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 168 OUT2 304 |
| ✓ | BbhGeometricShapes82Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 171 |
| ✓ | BbhGeometricShapes83Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 322 |
| ✓ | BbhGeometricShapes84Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 098 |
| ✓ | BbhGeometricShapes85BmThis SVG path element <path d="M 0.58,63.64 L 34.42,42.20"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT948 |
| ✓ | BbhGeometricShapes86Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 242 |
| × | BbhGeometricShapes87Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes87Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 084 OUT3 899 |
| ✓ | BbhGeometricShapes88Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT2 445 |
| ✓ | BbhGeometricShapes89Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 677 |
| ✓ | BbhGeometricShapes90Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT2 249 |
| ✓ | BbhGeometricShapes91Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 007 |
| ✓ | BbhGeometricShapes92Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 228 |
| × | BbhGeometricShapes93Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes93Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 084 OUT4 008 |
| ✓ | BbhGeometricShapes94Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 584 |
| × | BbhGeometricShapes95Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes95Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT2 655 |
| ✓ | BbhGeometricShapes96Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 960 |
| ✓ | BbhGeometricShapes97Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT2 245 |
| × | BbhGeometricShapes98Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes98Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 084 OUT1 486 |
| ✓ | BbhGeometricShapes99Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 550 |
Reasoning - BBH geometric shapes (SVG paths) (set 4)
25 passed7 failed |
78.1% |
IN35 266 OUT66 609 | |
| ✓ | BbhGeometricShapes105Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 164 OUT2 229 |
| ✓ | BbhGeometricShapes106Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 324 |
| ✓ | BbhGeometricShapes107BmThis SVG path element <path d="M 22.00,62.00 L 30.00,74.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT748 |
| ✓ | BbhGeometricShapes108Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 825 |
| × | BbhGeometricShapes109Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes109Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT2 214 |
| × | BbhGeometricShapes110Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes110Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 097 OUT1 655 |
| × | BbhGeometricShapes111Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes111Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 084 OUT2 786 |
| ✓ | BbhGeometricShapes112Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 274 |
| ✓ | BbhGeometricShapes113Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 141 OUT3 703 |
| ✓ | BbhGeometricShapes114Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT2 877 |
| ✓ | BbhGeometricShapes115Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT1 261 |
| ✓ | BbhGeometricShapes116Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 109 |
| ✓ | BbhGeometricShapes117Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 181 OUT2 435 |
| × | BbhGeometricShapes118Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes118Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 097 OUT3 108 |
| ✓ | BbhGeometricShapes119BmThis SVG path element <path d="M 17.11,37.49 L 43.48,13.53"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT863 |
| × | BbhGeometricShapes120Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes120Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 110 OUT3 028 |
| ✓ | BbhGeometricShapes121Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 809 |
| ✓ | BbhGeometricShapes122Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 369 |
| ✓ | BbhGeometricShapes123BmThis SVG path element <path d="M 51.00,18.00 L 5.00,62.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT898 |
| ✓ | BbhGeometricShapes124Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT3 435 |
| ✓ | BbhGeometricShapes125Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT2 505 |
| ✓ | BbhGeometricShapes126Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT2 732 |
| ✓ | BbhGeometricShapes127Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 683 |
| × | BbhGeometricShapes128Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes128Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 097 OUT1 887 |
| ✓ | BbhGeometricShapes129Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 162 OUT1 799 |
| ✓ | BbhGeometricShapes130Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 702 |
| ✓ | BbhGeometricShapes131Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 207 OUT1 954 |
| ✓ | BbhGeometricShapes132BmThis SVG path element <path d="M 26.00,23.00 L 19.00,16.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT857 |
| ✓ | BbhGeometricShapes133Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT1 362 |
| × | BbhGeometricShapes134Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes134Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 084 OUT2 896 |
| ✓ | BbhGeometricShapes135Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 548 |
| ✓ | BbhGeometricShapes136BmThis SVG path element <path d="M 46.92,78.61 L 46.94,45.83"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT734 |
Reasoning - BBH geometric shapes (SVG paths) (set 5)
26 passed5 failed |
83.9% |
IN34 158 OUT68 821 | |
| ✓ | BbhGeometricShapes137Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT1 966 |
| ✓ | BbhGeometricShapes138Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT4 785 |
| ✓ | BbhGeometricShapes139BmThis SVG path element <path d="M 58.33,52.18 L 34.28,31.10"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT776 |
| ✓ | BbhGeometricShapes140Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT3 318 |
| ✓ | BbhGeometricShapes141BmThis SVG path element <path d="M 38.35,49.41 L 31.18,9.15"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT446 |
| × | BbhGeometricShapes142Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes142Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 097 OUT1 701 |
| ✓ | BbhGeometricShapes143Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 663 |
| ✓ | BbhGeometricShapes144Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 089 |
| × | BbhGeometricShapes145Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes145Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT2 583 |
| ✓ | BbhGeometricShapes146Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 403 |
| ✓ | BbhGeometricShapes147Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 227 |
| ✓ | BbhGeometricShapes148Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 498 |
| ✓ | BbhGeometricShapes149Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 788 |
| ✓ | BbhGeometricShapes150BmThis SVG path element <path d="M 90.41,82.38 L 45.53,97.84"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT616 |
| ✓ | BbhGeometricShapes151Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT2 859 |
| ✓ | BbhGeometricShapes152Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 782 |
| ✓ | BbhGeometricShapes153Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 208 |
| ✓ | BbhGeometricShapes154Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 905 |
| ✓ | BbhGeometricShapes155Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 788 |
| ✓ | BbhGeometricShapes156Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT3 079 |
| ✓ | BbhGeometricShapes157Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT3 898 |
| ✓ | BbhGeometricShapes158Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT2 179 |
| ✓ | BbhGeometricShapes159Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT2 527 |
| ✓ | BbhGeometricShapes160Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT2 076 |
| × | BbhGeometricShapes161Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes161Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT2 325 |
| ✓ | BbhGeometricShapes162Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 181 OUT2 654 |
| × | BbhGeometricShapes163Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes163Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 123 OUT3 100 |
| × | BbhGeometricShapes164Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes164Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 097 OUT2 718 |
| ✓ | BbhGeometricShapes165Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 461 |
| ✓ | BbhGeometricShapes166Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 336 |
| ✓ | BbhGeometricShapes167Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 168 OUT2 067 |
Reasoning - BBH geometric shapes (SVG paths) (set 6)
28 passed3 failed |
90.3% |
IN34 272 OUT73 862 | |
| ✓ | BbhGeometricShapes168Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 815 |
| ✓ | BbhGeometricShapes169Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 648 |
| ✓ | BbhGeometricShapes170Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT2 237 |
| ✓ | BbhGeometricShapes171Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 470 |
| ✓ | BbhGeometricShapes172Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT2 476 |
| ✓ | BbhGeometricShapes173Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT1 925 |
| ✓ | BbhGeometricShapes174Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 837 |
| ✓ | BbhGeometricShapes175Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT3 880 |
| ✓ | BbhGeometricShapes176Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 762 |
| ✓ | BbhGeometricShapes177Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT3 514 |
| ✓ | BbhGeometricShapes178Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT2 047 |
| ✓ | BbhGeometricShapes179Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 418 |
| ✓ | BbhGeometricShapes180Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 533 |
| × | BbhGeometricShapes181Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes181Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT2 139 |
| × | BbhGeometricShapes182Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes182Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 084 OUT3 026 |
| ✓ | BbhGeometricShapes183Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT4 232 |
| ✓ | BbhGeometricShapes184BmThis SVG path element <path d="M 25.00,38.00 L 89.00,58.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT591 |
| ✓ | BbhGeometricShapes185Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT2 100 |
| ✓ | BbhGeometricShapes186Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 979 |
| ✓ | BbhGeometricShapes187BmThis SVG path element <path d="M 51.78,63.32 L 1.41,97.49"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT614 |
| ✓ | BbhGeometricShapes188Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT2 674 |
| ✓ | BbhGeometricShapes189Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 920 |
| ✓ | BbhGeometricShapes190Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 168 OUT2 467 |
| ✓ | BbhGeometricShapes191Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 168 OUT1 701 |
| ✓ | BbhGeometricShapes192Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT2 240 |
| ✓ | BbhGeometricShapes193BmThis SVG path element <path d="M 41.63,86.23 L 86.15,83.17"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT493 |
| ✓ | BbhGeometricShapes194Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT5 148 |
| ✓ | BbhGeometricShapes195Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT2 505 |
| ✓ | BbhGeometricShapes196Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 880 |
| ✓ | BbhGeometricShapes197Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 776 |
| × | BbhGeometricShapes198Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(I)")
Context: BbhGeometricShapes198Bm: expected (I), got [j]
Values:
got = "j" |
Failed | IN1 087 OUT4 815 |
Reasoning - BBH geometric shapes (SVG paths) (set 7)
23 passed9 failed |
71.9% |
IN35 108 OUT70 535 | |
| × | BbhGeometricShapes199BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes199BN: expected (K), got [(H)] |
Failed | IN1 084 OUT3 122 |
| × | BbhGeometricShapes200BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes200BN: expected (K), got [(A)] |
Failed | IN1 099 OUT2 323 |
| ✓ | BbhGeometricShapes201BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 564 |
| ✓ | BbhGeometricShapes202BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT1 105 |
| × | BbhGeometricShapes203BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes203BN: expected (K), got [(A)] |
Failed | IN1 097 OUT1 621 |
| ✓ | BbhGeometricShapes204BNThis SVG path element <path d="M 68.00,36.00 L 61.00,80.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT719 |
| ✓ | BbhGeometricShapes205BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 139 OUT3 154 |
| ✓ | BbhGeometricShapes206BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT1 567 |
| ✓ | BbhGeometricShapes207BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT1 802 |
| ✓ | BbhGeometricShapes208BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 377 |
| ✓ | BbhGeometricShapes209BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT2 036 |
| ✓ | BbhGeometricShapes210BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT3 399 |
| × | BbhGeometricShapes211BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes211BN: expected (K), got [(H)] |
Failed | IN1 123 OUT3 604 |
| ✓ | BbhGeometricShapes212BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT2 252 |
| ✓ | BbhGeometricShapes213BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT1 930 |
| ✓ | BbhGeometricShapes214BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 893 |
| ✓ | BbhGeometricShapes215BNThis SVG path element <path d="M 37.71,30.46 L 40.10,45.62"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT744 |
| ✓ | BbhGeometricShapes216BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT2 364 |
| ✓ | BbhGeometricShapes217BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 211 |
| ✓ | BbhGeometricShapes218BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 328 |
| ✓ | BbhGeometricShapes219BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 226 |
| × | BbhGeometricShapes220BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes220BN: expected (K), got [(A)] |
Failed | IN1 097 OUT1 942 |
| × | BbhGeometricShapes221BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes221BN: expected (K), got [(A)] |
Failed | IN1 101 OUT2 972 |
| ✓ | BbhGeometricShapes222BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 866 |
| × | BbhGeometricShapes223BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes223BN: expected (K), got [(H)] |
Failed | IN1 097 OUT4 943 |
| × | BbhGeometricShapes224BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes224BN: expected (K), got [(H)] |
Failed | IN1 084 OUT1 936 |
| ✓ | BbhGeometricShapes225BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT2 727 |
| × | BbhGeometricShapes226BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes226BN: expected (K), got [(H)] |
Failed | IN1 084 OUT3 047 |
| ✓ | BbhGeometricShapes227BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT1 802 |
| ✓ | BbhGeometricShapes228BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT1 533 |
| ✓ | BbhGeometricShapes229BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 167 |
| ✓ | BbhGeometricShapes230BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 259 |
Reasoning - BBH geometric shapes (SVG paths) (set 8)
29 passed3 failed |
90.6% |
IN33 687 OUT48 382 | |
| ✓ | BbhGeometricShapes231BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 589 |
| ✓ | BbhGeometricShapes232BNThis SVG path element <path d="M 93.43,69.32 L 68.06,88.58"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT764 |
| ✓ | BbhGeometricShapes233BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT1 899 |
| ✓ | BbhGeometricShapes234BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 361 |
| ✓ | BbhGeometricShapes235BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 872 |
| ✓ | BbhGeometricShapes236BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT3 230 |
| ✓ | BbhGeometricShapes237BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 481 |
| × | BbhGeometricShapes238BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes238BN: expected (K), got [(H)] |
Failed | IN1 084 OUT1 222 |
| ✓ | BbhGeometricShapes239BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 656 |
| ✓ | BbhGeometricShapes240BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 685 |
| ✓ | BbhGeometricShapes241BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT1 934 |
| × | BbhGeometricShapes242BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes242BN: expected (K), got [(A)] |
Failed | IN1 101 OUT2 838 |
| ✓ | BbhGeometricShapes243BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT2 093 |
| ✓ | BbhGeometricShapes244BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 275 |
| ✓ | BbhGeometricShapes245BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 445 |
| ✓ | BbhGeometricShapes246BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 853 |
| × | BbhGeometricShapes247BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes247BN: expected (K), got [(H)] |
Failed | IN1 093 OUT3 638 |
| ✓ | BbhGeometricShapes248BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT2 509 |
| ✓ | BbhGeometricShapes249BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT3 091 |
| ✓ | BbhHyperbaton10BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT879 |
| ✓ | BbhHyperbaton11BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT660 |
| ✓ | BbhHyperbaton12BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 114 |
| ✓ | BbhHyperbaton13BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT597 |
| ✓ | BbhHyperbaton14BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT247 |
| ✓ | BbhHyperbaton15BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT860 |
| ✓ | BbhHyperbaton16BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT573 |
| ✓ | BbhHyperbaton17BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT505 |
| ✓ | BbhHyperbaton18BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 022 |
| ✓ | BbhHyperbaton19BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 017 |
| ✓ | BbhHyperbaton20BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT833 |
| ✓ | BbhHyperbaton8BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT768 |
| ✓ | BbhHyperbaton9BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT872 |
Reasoning - BBH hyperbaton - adjective ordering (set 1)
32 passed |
100.0% |
IN31 442 OUT27 683 | |
| ✓ | BbhHyperbaton21BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT597 |
| ✓ | BbhHyperbaton22BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 576 |
| ✓ | BbhHyperbaton23BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 295 |
| ✓ | BbhHyperbaton24BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT436 |
| ✓ | BbhHyperbaton25BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT736 |
| ✓ | BbhHyperbaton26BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT476 |
| ✓ | BbhHyperbaton27BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 081 |
| ✓ | BbhHyperbaton28BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT898 |
| ✓ | BbhHyperbaton29BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 007 |
| ✓ | BbhHyperbaton30BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT902 |
| ✓ | BbhHyperbaton31BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 134 |
| ✓ | BbhHyperbaton32BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT583 |
| ✓ | BbhHyperbaton33BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT576 |
| ✓ | BbhHyperbaton34BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT527 |
| ✓ | BbhHyperbaton35BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT892 |
| ✓ | BbhHyperbaton36BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT739 |
| ✓ | BbhHyperbaton37BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT697 |
| ✓ | BbhHyperbaton38BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 141 |
| ✓ | BbhHyperbaton39BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT554 |
| ✓ | BbhHyperbaton40BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 043 |
| ✓ | BbhHyperbaton41BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT812 |
| ✓ | BbhHyperbaton42BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT499 |
| ✓ | BbhHyperbaton43BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT865 |
| ✓ | BbhHyperbaton44BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 299 |
| ✓ | BbhHyperbaton45BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT933 |
| ✓ | BbhHyperbaton46BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT839 |
| ✓ | BbhHyperbaton47BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT993 |
| ✓ | BbhHyperbaton48BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 076 |
| ✓ | BbhHyperbaton49BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT771 |
| ✓ | BbhHyperbaton50BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT863 |
| ✓ | BbhHyperbaton51BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT575 |
| ✓ | BbhHyperbaton52BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 268 |
Reasoning - BBH hyperbaton - adjective ordering (set 2)
32 passed |
100.0% |
IN31 460 OUT29 945 | |
| ✓ | BbhHyperbaton53BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT614 |
| ✓ | BbhHyperbaton54BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 063 |
| ✓ | BbhHyperbaton55BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 026 |
| ✓ | BbhHyperbaton56BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT750 |
| ✓ | BbhHyperbaton57BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT997 |
| ✓ | BbhHyperbaton58BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 167 |
| ✓ | BbhHyperbaton59BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 253 |
| ✓ | BbhHyperbaton60BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT990 |
| ✓ | BbhHyperbaton61BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT374 |
| ✓ | BbhHyperbaton62BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT370 |
| ✓ | BbhHyperbaton63BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT623 |
| ✓ | BbhHyperbaton64BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT827 |
| ✓ | BbhHyperbaton65BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT1 216 |
| ✓ | BbhHyperbaton66BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 159 |
| ✓ | BbhHyperbaton67BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 499 |
| ✓ | BbhHyperbaton68BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT740 |
| ✓ | BbhHyperbaton69BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT861 |
| ✓ | BbhHyperbaton70BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT567 |
| ✓ | BbhHyperbaton71BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 613 |
| ✓ | BbhHyperbaton72BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT947 |
| ✓ | BbhHyperbaton73BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT822 |
| ✓ | BbhHyperbaton74BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT915 |
| ✓ | BbhHyperbaton75BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT811 |
| ✓ | BbhHyperbaton76BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT948 |
| ✓ | BbhHyperbaton77BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT930 |
| ✓ | BbhHyperbaton78BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT790 |
| ✓ | BbhHyperbaton79BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT885 |
| ✓ | BbhHyperbaton80BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT982 |
| ✓ | BbhHyperbaton81BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT916 |
| ✓ | BbhHyperbaton82BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT870 |
| ✓ | BbhHyperbaton83BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 102 |
| ✓ | BbhHyperbaton84BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 318 |
Reasoning - BBH hyperbaton - adjective ordering (set 3)
30 passed |
100.0% |
IN29 494 OUT23 392 | |
| ✓ | BbhHyperbaton100BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT801 |
| ✓ | BbhHyperbaton101BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 169 |
| ✓ | BbhHyperbaton102BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT853 |
| ✓ | BbhHyperbaton103BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT696 |
| ✓ | BbhHyperbaton104BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT396 |
| ✓ | BbhHyperbaton105BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT863 |
| ✓ | BbhHyperbaton106BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 064 |
| ✓ | BbhHyperbaton107BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 007 |
| ✓ | BbhHyperbaton108BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT483 |
| ✓ | BbhHyperbaton109BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT552 |
| ✓ | BbhHyperbaton110BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT949 |
| ✓ | BbhHyperbaton111BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT605 |
| ✓ | BbhHyperbaton112BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT974 |
| ✓ | BbhHyperbaton113BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT489 |
| ✓ | BbhHyperbaton114BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT666 |
| ✓ | BbhHyperbaton85BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 259 |
| ✓ | BbhHyperbaton86BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT889 |
| ✓ | BbhHyperbaton87BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT597 |
| ✓ | BbhHyperbaton88BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT889 |
| ✓ | BbhHyperbaton89BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT659 |
| ✓ | BbhHyperbaton90BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT707 |
| ✓ | BbhHyperbaton91BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT956 |
| ✓ | BbhHyperbaton92BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT486 |
| ✓ | BbhHyperbaton93BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT496 |
| ✓ | BbhHyperbaton94BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT677 |
| ✓ | BbhHyperbaton95BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT898 |
| ✓ | BbhHyperbaton96BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 383 |
| ✓ | BbhHyperbaton97BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT896 |
| ✓ | BbhHyperbaton98BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT414 |
| ✓ | BbhHyperbaton99BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT619 |
Reasoning - BBH hyperbaton - adjective ordering (set 4)
32 passed |
100.0% |
IN31 426 OUT28 504 | |
| ✓ | BbhHyperbaton115BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 070 |
| ✓ | BbhHyperbaton116BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT797 |
| ✓ | BbhHyperbaton117BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT722 |
| ✓ | BbhHyperbaton118BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT831 |
| ✓ | BbhHyperbaton119BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 149 |
| ✓ | BbhHyperbaton120BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT480 |
| ✓ | BbhHyperbaton121BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT922 |
| ✓ | BbhHyperbaton122BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT998 |
| ✓ | BbhHyperbaton123BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT849 |
| ✓ | BbhHyperbaton124BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT982 |
| ✓ | BbhHyperbaton125BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT455 |
| ✓ | BbhHyperbaton126BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 757 |
| ✓ | BbhHyperbaton127BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 024 |
| ✓ | BbhHyperbaton128BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT371 |
| ✓ | BbhHyperbaton129BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT661 |
| ✓ | BbhHyperbaton130BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT671 |
| ✓ | BbhHyperbaton131BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT878 |
| ✓ | BbhHyperbaton132BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 637 |
| ✓ | BbhHyperbaton133BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT964 |
| ✓ | BbhHyperbaton134BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 000 |
| ✓ | BbhHyperbaton135BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT319 |
| ✓ | BbhHyperbaton136BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT682 |
| ✓ | BbhHyperbaton137BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 012 |
| ✓ | BbhHyperbaton138BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT774 |
| ✓ | BbhHyperbaton139BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 115 |
| ✓ | BbhHyperbaton140BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 100 |
| ✓ | BbhHyperbaton141BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT976 |
| ✓ | BbhHyperbaton142BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT709 |
| ✓ | BbhHyperbaton143BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT740 |
| ✓ | BbhHyperbaton144BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 107 |
| ✓ | BbhHyperbaton145BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT331 |
| ✓ | BbhHyperbaton146BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT1 421 |
Reasoning - BBH hyperbaton - adjective ordering (set 5)
32 passed |
100.0% |
IN31 448 OUT29 977 | |
| ✓ | BbhHyperbaton147BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT968 |
| ✓ | BbhHyperbaton148BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT681 |
| ✓ | BbhHyperbaton149BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 144 |
| ✓ | BbhHyperbaton150BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 119 |
| ✓ | BbhHyperbaton151BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 070 |
| ✓ | BbhHyperbaton152BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 541 |
| ✓ | BbhHyperbaton153BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT775 |
| ✓ | BbhHyperbaton154BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT804 |
| ✓ | BbhHyperbaton155BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 006 |
| ✓ | BbhHyperbaton156BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT589 |
| ✓ | BbhHyperbaton157BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT621 |
| ✓ | BbhHyperbaton158BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 835 |
| ✓ | BbhHyperbaton159BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT884 |
| ✓ | BbhHyperbaton160BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT328 |
| ✓ | BbhHyperbaton161BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 843 |
| ✓ | BbhHyperbaton162BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 648 |
| ✓ | BbhHyperbaton163BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT539 |
| ✓ | BbhHyperbaton164BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT857 |
| ✓ | BbhHyperbaton165BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT1 134 |
| ✓ | BbhHyperbaton166BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT999 |
| ✓ | BbhHyperbaton167BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT833 |
| ✓ | BbhHyperbaton168BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT591 |
| ✓ | BbhHyperbaton169BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT520 |
| ✓ | BbhHyperbaton170BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 106 |
| ✓ | BbhHyperbaton171BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 193 |
| ✓ | BbhHyperbaton172BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT847 |
| ✓ | BbhHyperbaton173BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT638 |
| ✓ | BbhHyperbaton174BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 011 |
| ✓ | BbhHyperbaton175BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT784 |
| ✓ | BbhHyperbaton176BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT511 |
| ✓ | BbhHyperbaton177BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT916 |
| ✓ | BbhHyperbaton178BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT642 |
Reasoning - BBH hyperbaton - adjective ordering (set 6)
32 passed |
100.0% |
IN31 436 OUT28 597 | |
| ✓ | BbhHyperbaton179BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT2 079 |
| ✓ | BbhHyperbaton180BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 469 |
| ✓ | BbhHyperbaton181BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT922 |
| ✓ | BbhHyperbaton182BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT547 |
| ✓ | BbhHyperbaton183BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT882 |
| ✓ | BbhHyperbaton184BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT802 |
| ✓ | BbhHyperbaton185BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT722 |
| ✓ | BbhHyperbaton186BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 385 |
| ✓ | BbhHyperbaton187BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT781 |
| ✓ | BbhHyperbaton188BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT724 |
| ✓ | BbhHyperbaton189BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT1 285 |
| ✓ | BbhHyperbaton190BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT520 |
| ✓ | BbhHyperbaton191BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT657 |
| ✓ | BbhHyperbaton192BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT502 |
| ✓ | BbhHyperbaton193BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT880 |
| ✓ | BbhHyperbaton194BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT530 |
| ✓ | BbhHyperbaton195BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 112 |
| ✓ | BbhHyperbaton196BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT931 |
| ✓ | BbhHyperbaton197BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT670 |
| ✓ | BbhHyperbaton198BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 642 |
| ✓ | BbhHyperbaton199BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT629 |
| ✓ | BbhHyperbaton200BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT531 |
| ✓ | BbhHyperbaton201BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT903 |
| ✓ | BbhHyperbaton202BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT813 |
| ✓ | BbhHyperbaton203BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT770 |
| ✓ | BbhHyperbaton204BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT448 |
| ✓ | BbhHyperbaton205BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT571 |
| ✓ | BbhHyperbaton206BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 451 |
| ✓ | BbhHyperbaton207BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 035 |
| ✓ | BbhHyperbaton208BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT788 |
| ✓ | BbhHyperbaton209BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT784 |
| ✓ | BbhHyperbaton210BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT832 |
Reasoning - BBH hyperbaton - adjective ordering (set 7)
32 passed |
100.0% |
IN31 422 OUT29 177 | |
| ✓ | BbhHyperbaton211BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT803 |
| ✓ | BbhHyperbaton212BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT819 |
| ✓ | BbhHyperbaton213BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 140 |
| ✓ | BbhHyperbaton214BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 479 |
| ✓ | BbhHyperbaton215BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 574 |
| ✓ | BbhHyperbaton216BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT1 038 |
| ✓ | BbhHyperbaton217BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT953 |
| ✓ | BbhHyperbaton218BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT888 |
| ✓ | BbhHyperbaton219BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT933 |
| ✓ | BbhHyperbaton220BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT623 |
| ✓ | BbhHyperbaton221BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT750 |
| ✓ | BbhHyperbaton222BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 033 |
| ✓ | BbhHyperbaton223BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 083 |
| ✓ | BbhHyperbaton224BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 178 |
| ✓ | BbhHyperbaton225BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 546 |
| ✓ | BbhHyperbaton226BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT725 |
| ✓ | BbhHyperbaton227BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT466 |
| ✓ | BbhHyperbaton228BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 306 |
| ✓ | BbhHyperbaton229BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 357 |
| ✓ | BbhHyperbaton230BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 328 |
| ✓ | BbhHyperbaton231BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT445 |
| ✓ | BbhHyperbaton232BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT657 |
| ✓ | BbhHyperbaton233BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT460 |
| ✓ | BbhHyperbaton234BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT529 |
| ✓ | BbhHyperbaton235BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT512 |
| ✓ | BbhHyperbaton236BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT589 |
| ✓ | BbhHyperbaton237BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT788 |
| ✓ | BbhHyperbaton238BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT437 |
| ✓ | BbhHyperbaton239BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT563 |
| ✓ | BbhHyperbaton240BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT839 |
| ✓ | BbhHyperbaton241BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT971 |
| ✓ | BbhHyperbaton242BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 365 |
Reasoning - BBH logic: boolean expressions, causal judgement, dates
22 passed2 failed |
91.7% |
IN25 847 OUT32 931 | |
| ✓ | BbhBooleanExpressionsBa0not ( True ) and ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT589 |
| ✓ | BbhBooleanExpressionsBa1True and not not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT732 |
| ✓ | BbhBooleanExpressionsBa2not True or False or ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT712 |
| ✓ | BbhBooleanExpressionsBa3False or not ( True ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT693 |
| ✓ | BbhBooleanExpressionsBa4True or not False and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT765 |
| ✓ | BbhBooleanExpressionsBa5False or not not not False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT1 196 |
| ✓ | BbhBooleanExpressionsBa6not True and ( False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT516 |
| ✓ | BbhBooleanExpressionsBa7True and not False or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN962 OUT762 |
| ✓ | BbhCausalJudgementBa0judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 975 |
| ✓ | BbhCausalJudgementBa1judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 348 OUT2 412 |
| ✓ | BbhCausalJudgementBa2judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 350 OUT1 739 |
| ✓ | BbhCausalJudgementBa3judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 343 OUT4 216 |
| ✓ | BbhCausalJudgementBa4judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT1 610 |
| ✓ | BbhCausalJudgementBa5judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT2 240 |
| ✓ | BbhCausalJudgementBa6judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 156 |
| ✓ | BbhCausalJudgementBa7judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT1 153 |
| ✓ | BbhDateUnderstandingBa0date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT590 |
| ✓ | BbhDateUnderstandingBa1date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 686 |
| × | BbhDateUnderstandingBa2date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(B)")
Context: BbhDateUnderstandingBa2: expected (B), got [(C)] |
Failed | IN1 077 OUT694 |
| ✓ | BbhDateUnderstandingBa3date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT1 381 |
| ✓ | BbhDateUnderstandingBa4date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT1 141 |
| ✓ | BbhDateUnderstandingBa5date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 080 |
| × | BbhDateUnderstandingBa6date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhDateUnderstandingBa6: expected (A), got [(D)] |
Failed | IN1 085 OUT3 274 |
| ✓ | BbhDateUnderstandingBa7Tomorrow is 11/12/2019. What is the date yesterday in MM/DD/YYYY?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT619 |
Reasoning - BBH logic: disambiguation, dyck languages, formal fallacies
21 passed3 failed |
87.5% |
IN24 577 OUT47 606 | |
| ✓ | BbhDisambiguationQaBa0resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT672 |
| ✓ | BbhDisambiguationQaBa1resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT1 276 |
| × | BbhDisambiguationQaBa2resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhDisambiguationQaBa2: expected (C), got [(B)] |
Failed | IN1 024 OUT1 607 |
| ✓ | BbhDisambiguationQaBa3resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT1 670 |
| ✓ | BbhDisambiguationQaBa4resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT939 |
| × | BbhDisambiguationQaBa5resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhDisambiguationQaBa5: expected (C), got [(B)] |
Failed | IN1 021 OUT2 118 |
| × | BbhDisambiguationQaBa6resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhDisambiguationQaBa6: expected (C), got [(A)] |
Failed | IN1 025 OUT986 |
| ✓ | BbhDisambiguationQaBa7resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 421 |
| ✓ | BbhDyckLanguagesBa0correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT466 |
| ✓ | BbhDyckLanguagesBa1correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT1 417 |
| ✓ | BbhDyckLanguagesBa2correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 036 OUT9 922 |
| ✓ | BbhDyckLanguagesBa3correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 607 |
| ✓ | BbhDyckLanguagesBa4correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT3 637 |
| ✓ | BbhDyckLanguagesBa5correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT2 212 |
| ✓ | BbhDyckLanguagesBa6correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 297 |
| ✓ | BbhDyckLanguagesBa7correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT2 314 |
| ✓ | BbhFormalFallaciesBa0decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT1 212 |
| ✓ | BbhFormalFallaciesBa1decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT1 098 |
| ✓ | BbhFormalFallaciesBa2decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT2 689 |
| ✓ | BbhFormalFallaciesBa3decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT2 750 |
| ✓ | BbhFormalFallaciesBa4decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 626 |
| ✓ | BbhFormalFallaciesBa5decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT1 607 |
| ✓ | BbhFormalFallaciesBa6decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 606 |
| ✓ | BbhFormalFallaciesBa7decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT1 457 |
Reasoning - BBH logic: geometric shapes, hyperbaton, logical deduction
23 passed1 failed |
95.8% |
IN25 418 OUT36 057 | |
| ✓ | BbhGeometricShapesBa0identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT1 982 |
| ✓ | BbhGeometricShapesBa1identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT1 248 |
| ✓ | BbhGeometricShapesBa2identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT3 452 |
| ✓ | BbhGeometricShapesBa3This SVG path element <path d="M 12.64,60.32 L 16.36,27.35"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT557 |
| × | BbhGeometricShapesBa4identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapesBa4: expected (K), got [(H)] |
Failed | IN1 110 OUT3 561 |
| ✓ | BbhGeometricShapesBa5identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT1 632 |
| ✓ | BbhGeometricShapesBa6identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 422 |
| ✓ | BbhGeometricShapesBa7identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT1 890 |
| ✓ | BbhHyperbatonBa0Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT587 |
| ✓ | BbhHyperbatonBa1Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 159 |
| ✓ | BbhHyperbatonBa2Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT2 474 |
| ✓ | BbhHyperbatonBa3Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT1 489 |
| ✓ | BbhHyperbatonBa4Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT808 |
| ✓ | BbhHyperbatonBa5Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT983 |
| ✓ | BbhHyperbatonBa6Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT815 |
| ✓ | BbhHyperbatonBa7Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT539 |
| ✓ | BbhLogicalDeductionFiveBa0order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT713 |
| ✓ | BbhLogicalDeductionFiveBa1order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 832 |
| ✓ | BbhLogicalDeductionFiveBa2order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT709 |
| ✓ | BbhLogicalDeductionFiveBa3order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 350 |
| ✓ | BbhLogicalDeductionFiveBa4order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 673 |
| ✓ | BbhLogicalDeductionFiveBa5order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT2 561 |
| ✓ | BbhLogicalDeductionFiveBa6order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT1 493 |
| ✓ | BbhLogicalDeductionFiveBa7order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 128 |
Reasoning - BBH logic: logical deduction, movie recommendation
24 passed |
100.0% |
IN25 463 OUT36 847 | |
| ✓ | BbhLogicalDeductionSeven0order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 344 |
| ✓ | BbhLogicalDeductionSeven1order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT2 765 |
| ✓ | BbhLogicalDeductionSeven2order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 824 |
| ✓ | BbhLogicalDeductionSeven3order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 787 |
| ✓ | BbhLogicalDeductionSeven4order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT1 519 |
| ✓ | BbhLogicalDeductionSeven5order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 095 OUT1 555 |
| ✓ | BbhLogicalDeductionSeven6order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 638 |
| ✓ | BbhLogicalDeductionSeven7order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 140 OUT2 278 |
| ✓ | BbhLogicalDeductionThree0order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT962 |
| ✓ | BbhLogicalDeductionThree1order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT923 |
| ✓ | BbhLogicalDeductionThree2order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT619 |
| ✓ | BbhLogicalDeductionThree3order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT518 |
| ✓ | BbhLogicalDeductionThree4order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 047 |
| ✓ | BbhLogicalDeductionThree5order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT635 |
| ✓ | BbhLogicalDeductionThree6order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT837 |
| ✓ | BbhLogicalDeductionThree7order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT604 |
| ✓ | BbhMovieRec0Find a movie similar to Batman, The Mask, The Fugitive, Pretty Woman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 986 |
| ✓ | BbhMovieRec1pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 498 |
| ✓ | BbhMovieRec2pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 661 |
| ✓ | BbhMovieRec3pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 702 |
| ✓ | BbhMovieRec4Find a movie similar to Braveheart, The Mask, The Fugitive, Batman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT4 236 |
| ✓ | BbhMovieRec5pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 663 |
| ✓ | BbhMovieRec6pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 203 |
| ✓ | BbhMovieRec7pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT2 043 |
Reasoning - BBH logic: multistep arithmetic, navigation, object counting
24 passed |
100.0% |
IN23 760 OUT25 263 | |
| ✓ | BbhMultistepArith0((-1 + 2 + 9 * 5) - (-2 + -4 + -4 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT750 |
| ✓ | BbhMultistepArith1((-9 * -5 - 6 + -2) - (-8 - -6 * -3 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 257 |
| ✓ | BbhMultistepArith2((3 * -3 * 6 + -5) - (-2 + -7 - 7 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 078 |
| ✓ | BbhMultistepArith3((6 * -6 * 8 * 1) * (-1 * 7 * -6 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 165 |
| ✓ | BbhMultistepArith4((-6 - -4 + 9 + 0) + (1 + -4 - -9 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 605 |
| ✓ | BbhMultistepArith5((-6 - 4 * 2 - 6) + (1 + -2 * 1 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT869 |
| ✓ | BbhMultistepArith6((1 - 0 + 1 - 4) - (-3 * 1 - -6 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 931 |
| ✓ | BbhMultistepArith7((1 + 7 * -9 + -5) + (3 + -5 * 2 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT880 |
| ✓ | BbhNavigate0does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT2 228 |
| ✓ | BbhNavigate1does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT835 |
| ✓ | BbhNavigate2does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 023 |
| ✓ | BbhNavigate3does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT818 |
| ✓ | BbhNavigate4does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 841 |
| ✓ | BbhNavigate5does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT749 |
| ✓ | BbhNavigate6does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT2 268 |
| ✓ | BbhNavigate7does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT478 |
| ✓ | BbhObjectCount0count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT548 |
| ✓ | BbhObjectCount1count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 825 |
| ✓ | BbhObjectCount2I have a fridge, a chair, and a microwave. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT355 |
| ✓ | BbhObjectCount3count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT655 |
| ✓ | BbhObjectCount4count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT229 |
| ✓ | BbhObjectCount5I have a fish, a bear, and a frog. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT210 |
| ✓ | BbhObjectCount6I have a car, and a toaster. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT181 |
| ✓ | BbhObjectCount7count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT485 |
Reasoning - BBH logic: object tracking, web of lies, word sorting
23 passed1 failed |
95.8% |
IN24 382 OUT26 637 | |
| ✓ | BbhTrackThree0track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT830 |
| ✓ | BbhTrackThree1track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT893 |
| ✓ | BbhTrackThree2track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT652 |
| ✓ | BbhTrackThree3track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT856 |
| ✓ | BbhTrackThree4track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT934 |
| ✓ | BbhTrackThree5track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT711 |
| ✓ | BbhTrackThree6track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT871 |
| ✓ | BbhTrackThree7track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT888 |
| ✓ | BbhWebLies0evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 080 |
| ✓ | BbhWebLies1evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT900 |
| ✓ | BbhWebLies2evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 400 |
| ✓ | BbhWebLies3evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 482 |
| ✓ | BbhWebLies4evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 309 |
| ✓ | BbhWebLies5evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 210 |
| ✓ | BbhWebLies6evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 451 |
| ✓ | BbhWebLies7evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 099 |
| ✓ | BbhWordSort0Sort the following words alphabetically: List: syndrome therefrom1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN965 OUT480 |
| ✓ | BbhWordSort1sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT2 521 |
| ✓ | BbhWordSort2sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT897 |
| ✓ | BbhWordSort3sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT1 054 |
| ✓ | BbhWordSort4sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT597 |
| × | BbhWordSort5sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("apparition conference copra coupe dutton floruit ignore implement layperson messenger primitive superstitious turnoff westward")
Context: BbhWordSort5: expected sorted list, got [apparition conference copra coupe dutton floruit implement ignore layperson messenger primitive superstitious turnoff westward] |
Failed | IN985 OUT847 |
| ✓ | BbhWordSort6sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT2 289 |
| ✓ | BbhWordSort7sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT1 386 |
Reasoning - BBH logic: penguins table, colored objects, ruin names
22 passed2 failed |
91.7% |
IN25 960 OUT21 633 | |
| ✓ | BbhColoredObjects0reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT778 |
| ✓ | BbhColoredObjects1reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT652 |
| ✓ | BbhColoredObjects2reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT754 |
| ✓ | BbhColoredObjects3reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT979 |
| ✓ | BbhColoredObjects4reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT717 |
| ✓ | BbhColoredObjects5reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT1 108 |
| ✓ | BbhColoredObjects6reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT515 |
| ✓ | BbhColoredObjects7reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT532 |
| ✓ | BbhPenguins0answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 215 OUT468 |
| ✓ | BbhPenguins1answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT389 |
| ✓ | BbhPenguins2answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT639 |
| ✓ | BbhPenguins3answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 199 OUT1 086 |
| ✓ | BbhPenguins4answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT913 |
| ✓ | BbhPenguins5answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 205 OUT836 |
| ✓ | BbhPenguins6answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 224 OUT463 |
| ✓ | BbhPenguins7answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 200 OUT303 |
| ✓ | BbhRuinNames0pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 331 |
| × | BbhRuinNames1pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames1: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 009 OUT942 |
| ✓ | BbhRuinNames2pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT1 302 |
| ✓ | BbhRuinNames3pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 098 |
| × | BbhRuinNames4pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames4: expected (D), got [c]
Values:
got = "c" |
Failed | IN998 OUT1 639 |
| ✓ | BbhRuinNames5pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT1 896 |
| ✓ | BbhRuinNames6pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT485 |
| ✓ | BbhRuinNames7pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 808 |
Reasoning - BBH logic: temporal sequences, object tracking
24 passed |
100.0% |
IN27 179 OUT35 856 | |
| ✓ | BbhTemporal0find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT1 379 |
| ✓ | BbhTemporal1find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 463 |
| ✓ | BbhTemporal2find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT1 016 |
| ✓ | BbhTemporal3find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 121 OUT1 002 |
| ✓ | BbhTemporal4find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 046 |
| ✓ | BbhTemporal5find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 035 |
| ✓ | BbhTemporal6find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT1 066 |
| ✓ | BbhTemporal7find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT936 |
| ✓ | BbhTrackFive0track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 064 |
| ✓ | BbhTrackFive1track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 590 |
| ✓ | BbhTrackFive2track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 468 |
| ✓ | BbhTrackFive3track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 202 |
| ✓ | BbhTrackFive4track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 347 |
| ✓ | BbhTrackFive5track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 522 |
| ✓ | BbhTrackFive6track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT2 654 |
| ✓ | BbhTrackFive7track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 285 |
| ✓ | BbhTrackSeven0track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 547 |
| ✓ | BbhTrackSeven1track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 132 |
| ✓ | BbhTrackSeven2track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 621 |
| ✓ | BbhTrackSeven3track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT2 811 |
| ✓ | BbhTrackSeven4track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 170 OUT1 436 |
| ✓ | BbhTrackSeven5track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 700 |
| ✓ | BbhTrackSeven6track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 843 |
| ✓ | BbhTrackSeven7track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 691 |
Reasoning - BBH logic: translation errors, snarks, sports
19 passed5 failed |
79.2% |
IN25 213 OUT28 690 | |
| ✓ | BbhSalientTransErr0identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 186 OUT883 |
| × | BbhSalientTransErr1identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhSalientTransErr1: expected (A), got [(D)] |
Failed | IN1 186 OUT943 |
| × | BbhSalientTransErr2identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(F)")
Context: BbhSalientTransErr2: expected (F), got [(D)] |
Failed | IN1 183 OUT1 738 |
| × | BbhSalientTransErr3identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(F)")
Context: BbhSalientTransErr3: expected (F), got [(D)] |
Failed | IN1 180 OUT1 317 |
| ✓ | BbhSalientTransErr4identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 177 OUT2 769 |
| ✓ | BbhSalientTransErr5identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 189 OUT2 010 |
| ✓ | BbhSalientTransErr6identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 199 OUT887 |
| ✓ | BbhSalientTransErr7identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 164 OUT949 |
| ✓ | BbhSnarks0Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT277 |
| ✓ | BbhSnarks1Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 377 |
| ✓ | BbhSnarks2Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT801 |
| ✓ | BbhSnarks3Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT776 |
| ✓ | BbhSnarks4Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 781 |
| ✓ | BbhSnarks5Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT986 |
| ✓ | BbhSnarks6Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 146 |
| ✓ | BbhSnarks7Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 222 |
| × | BbhSports0Is the following sentence plausible? "Elias Lindholm beat the buzzer."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("no")
Context: BbhSports0: expected no, got [Yes] |
Failed | IN969 OUT1 647 |
| ✓ | BbhSports1judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN968 OUT1 411 |
| ✓ | BbhSports2judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN972 OUT1 070 |
| ✓ | BbhSports3judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN976 OUT677 |
| ✓ | BbhSports4judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN970 OUT813 |
| ✓ | BbhSports5judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN976 OUT1 317 |
| ✓ | BbhSports6Is the following sentence plausible? "David Silva took a throw in."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN967 OUT795 |
| × | BbhSports7judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("no")
Context: BbhSports7: expected no, got [False] |
Failed | IN970 OUT1 098 |
Reasoning - BBH logical deduction - order 3 objects (set 1)
32 passed |
100.0% |
IN33 587 OUT23 703 | |
| ✓ | BbhLogicalDeductionThree21BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT872 |
| ✓ | BbhLogicalDeductionThree22BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT391 |
| ✓ | BbhLogicalDeductionThree23BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT660 |
| ✓ | BbhLogicalDeductionThree24BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT761 |
| ✓ | BbhLogicalDeductionThree25BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT991 |
| ✓ | BbhLogicalDeductionThree26BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT655 |
| ✓ | BbhLogicalDeductionThree27BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT1 198 |
| ✓ | BbhLogicalDeductionThree28BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT846 |
| ✓ | BbhLogicalDeductionThree29BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT707 |
| ✓ | BbhLogicalDeductionThree30BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT866 |
| ✓ | BbhLogicalDeductionThree31BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT794 |
| ✓ | BbhLogicalDeductionThree32BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT919 |
| ✓ | BbhLogicalDeductionThree33BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT820 |
| ✓ | BbhLogicalDeductionThree34BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT1 354 |
| ✓ | BbhLogicalDeductionThree35BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT579 |
| ✓ | BbhLogicalDeductionThree36BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT814 |
| ✓ | BbhLogicalDeductionThree37BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT491 |
| ✓ | BbhLogicalDeductionThree38BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT481 |
| ✓ | BbhLogicalDeductionThree39BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT435 |
| ✓ | BbhLogicalDeductionThree40BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT634 |
| ✓ | BbhLogicalDeductionThree41BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT457 |
| ✓ | BbhLogicalDeductionThree42BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT632 |
| ✓ | BbhLogicalDeductionThree43BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT928 |
| ✓ | BbhLogicalDeductionThree44BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT626 |
| ✓ | BbhLogicalDeductionThree45BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT544 |
| ✓ | BbhLogicalDeductionThree46BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT1 167 |
| ✓ | BbhLogicalDeductionThree47BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT359 |
| ✓ | BbhLogicalDeductionThree48BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT1 000 |
| ✓ | BbhLogicalDeductionThree49BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 300 |
| ✓ | BbhLogicalDeductionThree50BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT404 |
| ✓ | BbhLogicalDeductionThree51BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT563 |
| ✓ | BbhLogicalDeductionThree52BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT455 |
Reasoning - BBH logical deduction - order 3 objects (set 2)
32 passed |
100.0% |
IN33 436 OUT23 676 | |
| ✓ | BbhLogicalDeductionThree53BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT1 137 |
| ✓ | BbhLogicalDeductionThree54BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT335 |
| ✓ | BbhLogicalDeductionThree55BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT849 |
| ✓ | BbhLogicalDeductionThree56BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT375 |
| ✓ | BbhLogicalDeductionThree57BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT431 |
| ✓ | BbhLogicalDeductionThree58BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT581 |
| ✓ | BbhLogicalDeductionThree59BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT417 |
| ✓ | BbhLogicalDeductionThree60BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT1 722 |
| ✓ | BbhLogicalDeductionThree61BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT376 |
| ✓ | BbhLogicalDeductionThree62BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 011 |
| ✓ | BbhLogicalDeductionThree63BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT720 |
| ✓ | BbhLogicalDeductionThree64BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT612 |
| ✓ | BbhLogicalDeductionThree65BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 480 |
| ✓ | BbhLogicalDeductionThree66BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT720 |
| ✓ | BbhLogicalDeductionThree67BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT369 |
| ✓ | BbhLogicalDeductionThree68BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT684 |
| ✓ | BbhLogicalDeductionThree69BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT751 |
| ✓ | BbhLogicalDeductionThree70BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT399 |
| ✓ | BbhLogicalDeductionThree71BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT915 |
| ✓ | BbhLogicalDeductionThree72BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT863 |
| ✓ | BbhLogicalDeductionThree73BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 033 |
| ✓ | BbhLogicalDeductionThree74BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT1 156 |
| ✓ | BbhLogicalDeductionThree75BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT422 |
| ✓ | BbhLogicalDeductionThree76BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT673 |
| ✓ | BbhLogicalDeductionThree77BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT560 |
| ✓ | BbhLogicalDeductionThree78BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT677 |
| ✓ | BbhLogicalDeductionThree79BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT752 |
| ✓ | BbhLogicalDeductionThree80BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT549 |
| ✓ | BbhLogicalDeductionThree81BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT805 |
| ✓ | BbhLogicalDeductionThree82BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT896 |
| ✓ | BbhLogicalDeductionThree83BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT594 |
| ✓ | BbhLogicalDeductionThree84BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT812 |
Reasoning - BBH logical deduction - order 3 objects (set 3)
32 passed |
100.0% |
IN33 510 OUT24 203 | |
| ✓ | BbhLogicalDeductionThree100BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT474 |
| ✓ | BbhLogicalDeductionThree101BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 061 |
| ✓ | BbhLogicalDeductionThree102BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT438 |
| ✓ | BbhLogicalDeductionThree103BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT492 |
| ✓ | BbhLogicalDeductionThree104BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT614 |
| ✓ | BbhLogicalDeductionThree105BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT902 |
| ✓ | BbhLogicalDeductionThree106BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT961 |
| ✓ | BbhLogicalDeductionThree107BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT1 086 |
| ✓ | BbhLogicalDeductionThree108BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT482 |
| ✓ | BbhLogicalDeductionThree109BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT979 |
| ✓ | BbhLogicalDeductionThree110BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT456 |
| ✓ | BbhLogicalDeductionThree111BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT724 |
| ✓ | BbhLogicalDeductionThree112BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT702 |
| ✓ | BbhLogicalDeductionThree113BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT1 015 |
| ✓ | BbhLogicalDeductionThree114BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT466 |
| ✓ | BbhLogicalDeductionThree115BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT807 |
| ✓ | BbhLogicalDeductionThree116BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT1 063 |
| ✓ | BbhLogicalDeductionThree85BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT525 |
| ✓ | BbhLogicalDeductionThree86BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 067 |
| ✓ | BbhLogicalDeductionThree87BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT310 |
| ✓ | BbhLogicalDeductionThree88BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT921 |
| ✓ | BbhLogicalDeductionThree89BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT1 185 |
| ✓ | BbhLogicalDeductionThree90BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT321 |
| ✓ | BbhLogicalDeductionThree91BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT579 |
| ✓ | BbhLogicalDeductionThree92BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT574 |
| ✓ | BbhLogicalDeductionThree93BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 536 |
| ✓ | BbhLogicalDeductionThree94BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT450 |
| ✓ | BbhLogicalDeductionThree95BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT373 |
| ✓ | BbhLogicalDeductionThree96BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT864 |
| ✓ | BbhLogicalDeductionThree97BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT926 |
| ✓ | BbhLogicalDeductionThree98BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 200 |
| ✓ | BbhLogicalDeductionThree99BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT650 |
Reasoning - BBH logical deduction - order 3 objects (set 4)
32 passed |
100.0% |
IN33 558 OUT24 741 | |
| ✓ | BbhLogicalDeductionThree117BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT892 |
| ✓ | BbhLogicalDeductionThree118BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT849 |
| ✓ | BbhLogicalDeductionThree119BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT652 |
| ✓ | BbhLogicalDeductionThree120BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT549 |
| ✓ | BbhLogicalDeductionThree121BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 386 |
| ✓ | BbhLogicalDeductionThree122BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT915 |
| ✓ | BbhLogicalDeductionThree123BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT489 |
| ✓ | BbhLogicalDeductionThree124BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT577 |
| ✓ | BbhLogicalDeductionThree125BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT698 |
| ✓ | BbhLogicalDeductionThree126BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT640 |
| ✓ | BbhLogicalDeductionThree127BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT834 |
| ✓ | BbhLogicalDeductionThree128BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT665 |
| ✓ | BbhLogicalDeductionThree129BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT731 |
| ✓ | BbhLogicalDeductionThree130BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT843 |
| ✓ | BbhLogicalDeductionThree131BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT476 |
| ✓ | BbhLogicalDeductionThree132BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT882 |
| ✓ | BbhLogicalDeductionThree133BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT550 |
| ✓ | BbhLogicalDeductionThree134BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT322 |
| ✓ | BbhLogicalDeductionThree135BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT392 |
| ✓ | BbhLogicalDeductionThree136BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT1 139 |
| ✓ | BbhLogicalDeductionThree137BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 072 |
| ✓ | BbhLogicalDeductionThree138BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT508 |
| ✓ | BbhLogicalDeductionThree139BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 060 |
| ✓ | BbhLogicalDeductionThree140BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT776 |
| ✓ | BbhLogicalDeductionThree141BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT704 |
| ✓ | BbhLogicalDeductionThree142BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT1 045 |
| ✓ | BbhLogicalDeductionThree143BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT801 |
| ✓ | BbhLogicalDeductionThree144BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT832 |
| ✓ | BbhLogicalDeductionThree145BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT1 235 |
| ✓ | BbhLogicalDeductionThree146BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 025 |
| ✓ | BbhLogicalDeductionThree147BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT609 |
| ✓ | BbhLogicalDeductionThree148BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT593 |
Reasoning - BBH logical deduction - order 3 objects (set 5)
30 passed |
100.0% |
IN31 400 OUT25 838 | |
| ✓ | BbhLogicalDeductionThree149BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT623 |
| ✓ | BbhLogicalDeductionThree150BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT462 |
| ✓ | BbhLogicalDeductionThree151BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT1 065 |
| ✓ | BbhLogicalDeductionThree152BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT963 |
| ✓ | BbhLogicalDeductionThree153BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 266 |
| ✓ | BbhLogicalDeductionThree154BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 435 |
| ✓ | BbhLogicalDeductionThree155BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 887 |
| ✓ | BbhLogicalDeductionThree156BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT737 |
| ✓ | BbhLogicalDeductionThree157BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT634 |
| ✓ | BbhLogicalDeductionThree158BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT490 |
| ✓ | BbhLogicalDeductionThree159BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT605 |
| ✓ | BbhLogicalDeductionThree160BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 112 |
| ✓ | BbhLogicalDeductionThree161BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT596 |
| ✓ | BbhLogicalDeductionThree162BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT830 |
| ✓ | BbhLogicalDeductionThree163BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT739 |
| ✓ | BbhLogicalDeductionThree164BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT867 |
| ✓ | BbhLogicalDeductionThree165BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT822 |
| ✓ | BbhLogicalDeductionThree166BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT616 |
| ✓ | BbhLogicalDeductionThree167BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT887 |
| ✓ | BbhLogicalDeductionThree168BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT593 |
| ✓ | BbhLogicalDeductionThree169BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT826 |
| ✓ | BbhLogicalDeductionThree170BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT935 |
| ✓ | BbhLogicalDeductionThree171BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT406 |
| ✓ | BbhLogicalDeductionThree172BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT965 |
| ✓ | BbhLogicalDeductionThree173BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT1 107 |
| ✓ | BbhLogicalDeductionThree174BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT894 |
| ✓ | BbhLogicalDeductionThree175BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT1 037 |
| ✓ | BbhLogicalDeductionThree176BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT842 |
| ✓ | BbhLogicalDeductionThree177BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT688 |
| ✓ | BbhLogicalDeductionThree178BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT909 |
Reasoning - BBH logical deduction - order 3 objects (set 6)
32 passed |
100.0% |
IN33 455 OUT24 288 | |
| ✓ | BbhLogicalDeductionThreeObjects179BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT1 816 |
| ✓ | BbhLogicalDeductionThreeObjects180BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT624 |
| ✓ | BbhLogicalDeductionThreeObjects181BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT424 |
| ✓ | BbhLogicalDeductionThreeObjects182BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT372 |
| ✓ | BbhLogicalDeductionThreeObjects183BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT775 |
| ✓ | BbhLogicalDeductionThreeObjects184BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT516 |
| ✓ | BbhLogicalDeductionThreeObjects185BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT311 |
| ✓ | BbhLogicalDeductionThreeObjects186BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT669 |
| ✓ | BbhLogicalDeductionThreeObjects187BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT447 |
| ✓ | BbhLogicalDeductionThreeObjects188BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT556 |
| ✓ | BbhLogicalDeductionThreeObjects189BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT414 |
| ✓ | BbhLogicalDeductionThreeObjects190BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT1 406 |
| ✓ | BbhLogicalDeductionThreeObjects191BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT654 |
| ✓ | BbhLogicalDeductionThreeObjects192BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT623 |
| ✓ | BbhLogicalDeductionThreeObjects193BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT464 |
| ✓ | BbhLogicalDeductionThreeObjects194BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT1 177 |
| ✓ | BbhLogicalDeductionThreeObjects195BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT764 |
| ✓ | BbhLogicalDeductionThreeObjects196BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 079 |
| ✓ | BbhLogicalDeductionThreeObjects197BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT664 |
| ✓ | BbhLogicalDeductionThreeObjects198BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT1 813 |
| ✓ | BbhLogicalDeductionThreeObjects199BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT509 |
| ✓ | BbhLogicalDeductionThreeObjects200BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT1 107 |
| ✓ | BbhLogicalDeductionThreeObjects201BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT758 |
| ✓ | BbhLogicalDeductionThreeObjects202BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT1 327 |
| ✓ | BbhLogicalDeductionThreeObjects203BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT607 |
| ✓ | BbhLogicalDeductionThreeObjects204BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT600 |
| ✓ | BbhLogicalDeductionThreeObjects205BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT370 |
| ✓ | BbhLogicalDeductionThreeObjects206BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT793 |
| ✓ | BbhLogicalDeductionThreeObjects207BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT581 |
| ✓ | BbhLogicalDeductionThreeObjects208BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT651 |
| ✓ | BbhLogicalDeductionThreeObjects209BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT576 |
| ✓ | BbhLogicalDeductionThreeObjects210BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT841 |
Reasoning - BBH logical deduction - order 3 objects (set 7)
32 passed |
100.0% |
IN33 488 OUT24 870 | |
| ✓ | BbhLogicalDeductionThreeObjects211BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT520 |
| ✓ | BbhLogicalDeductionThreeObjects212BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT590 |
| ✓ | BbhLogicalDeductionThreeObjects213BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT1 189 |
| ✓ | BbhLogicalDeductionThreeObjects214BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT931 |
| ✓ | BbhLogicalDeductionThreeObjects215BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT564 |
| ✓ | BbhLogicalDeductionThreeObjects216BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 314 |
| ✓ | BbhLogicalDeductionThreeObjects217BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT480 |
| ✓ | BbhLogicalDeductionThreeObjects218BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT631 |
| ✓ | BbhLogicalDeductionThreeObjects219BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT799 |
| ✓ | BbhLogicalDeductionThreeObjects220BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT1 002 |
| ✓ | BbhLogicalDeductionThreeObjects221BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT355 |
| ✓ | BbhLogicalDeductionThreeObjects222BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT738 |
| ✓ | BbhLogicalDeductionThreeObjects223BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT438 |
| ✓ | BbhLogicalDeductionThreeObjects224BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT331 |
| ✓ | BbhLogicalDeductionThreeObjects225BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT1 200 |
| ✓ | BbhLogicalDeductionThreeObjects226BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT1 119 |
| ✓ | BbhLogicalDeductionThreeObjects227BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT361 |
| ✓ | BbhLogicalDeductionThreeObjects228BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT1 203 |
| ✓ | BbhLogicalDeductionThreeObjects229BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT443 |
| ✓ | BbhLogicalDeductionThreeObjects230BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT478 |
| ✓ | BbhLogicalDeductionThreeObjects231BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 511 |
| ✓ | BbhLogicalDeductionThreeObjects232BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT809 |
| ✓ | BbhLogicalDeductionThreeObjects233BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT1 017 |
| ✓ | BbhLogicalDeductionThreeObjects234BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT562 |
| ✓ | BbhLogicalDeductionThreeObjects235BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT467 |
| ✓ | BbhLogicalDeductionThreeObjects236BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT757 |
| ✓ | BbhLogicalDeductionThreeObjects237BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT876 |
| ✓ | BbhLogicalDeductionThreeObjects238BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT995 |
| ✓ | BbhLogicalDeductionThreeObjects239BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT1 060 |
| ✓ | BbhLogicalDeductionThreeObjects240BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT760 |
| ✓ | BbhLogicalDeductionThreeObjects241BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT457 |
| ✓ | BbhLogicalDeductionThreeObjects242BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT913 |
Reasoning - BBH logical deduction - order 5 objects (set 1)
30 passed |
100.0% |
IN31 850 OUT40 487 | |
| ✓ | BbhHyperbaton243BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT940 |
| ✓ | BbhHyperbaton244BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 387 |
| ✓ | BbhHyperbaton245BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT501 |
| ✓ | BbhHyperbaton246BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT674 |
| ✓ | BbhHyperbaton247BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT606 |
| ✓ | BbhHyperbaton248BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 053 |
| ✓ | BbhHyperbaton249BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT833 |
| ✓ | BbhLogicalDeductionFiveObjects10BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 614 |
| ✓ | BbhLogicalDeductionFiveObjects11BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 486 |
| ✓ | BbhLogicalDeductionFiveObjects12BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 518 |
| ✓ | BbhLogicalDeductionFiveObjects13BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT2 991 |
| ✓ | BbhLogicalDeductionFiveObjects14BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 184 |
| ✓ | BbhLogicalDeductionFiveObjects15BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT801 |
| ✓ | BbhLogicalDeductionFiveObjects16BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 577 |
| ✓ | BbhLogicalDeductionFiveObjects17BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT806 |
| ✓ | BbhLogicalDeductionFiveObjects18BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 305 |
| ✓ | BbhLogicalDeductionFiveObjects19BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 761 |
| ✓ | BbhLogicalDeductionFiveObjects20BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 397 |
| ✓ | BbhLogicalDeductionFiveObjects21BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT2 033 |
| ✓ | BbhLogicalDeductionFiveObjects22BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 210 |
| ✓ | BbhLogicalDeductionFiveObjects23BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 901 |
| ✓ | BbhLogicalDeductionFiveObjects24BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 714 |
| ✓ | BbhLogicalDeductionFiveObjects25BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 384 |
| ✓ | BbhLogicalDeductionFiveObjects26BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 315 |
| ✓ | BbhLogicalDeductionFiveObjects27BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT1 795 |
| ✓ | BbhLogicalDeductionFiveObjects28BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT935 |
| ✓ | BbhLogicalDeductionFiveObjects29BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 000 |
| ✓ | BbhLogicalDeductionFiveObjects30BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 196 |
| ✓ | BbhLogicalDeductionFiveObjects8BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT2 163 |
| ✓ | BbhLogicalDeductionFiveObjects9BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 407 |
Reasoning - BBH logical deduction - order 5 objects (set 2)
32 passed |
100.0% |
IN34 885 OUT52 576 | |
| ✓ | BbhLogicalDeductionFive189BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 025 |
| ✓ | BbhLogicalDeductionFive190BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT966 |
| ✓ | BbhLogicalDeductionFive191BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 232 |
| ✓ | BbhLogicalDeductionFive192BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT1 170 |
| ✓ | BbhLogicalDeductionFive193BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 704 |
| ✓ | BbhLogicalDeductionFive194BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 209 |
| ✓ | BbhLogicalDeductionFive195BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT2 060 |
| ✓ | BbhLogicalDeductionFive196BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 132 |
| ✓ | BbhLogicalDeductionFive197BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 118 |
| ✓ | BbhLogicalDeductionFive198BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 400 |
| ✓ | BbhLogicalDeductionFive199BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT2 751 |
| ✓ | BbhLogicalDeductionFive200BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT2 137 |
| ✓ | BbhLogicalDeductionFive201BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 455 |
| ✓ | BbhLogicalDeductionFive202BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 252 |
| ✓ | BbhLogicalDeductionFive203BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT3 173 |
| ✓ | BbhLogicalDeductionFive204BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT956 |
| ✓ | BbhLogicalDeductionFive205BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT2 233 |
| ✓ | BbhLogicalDeductionFive206BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 647 |
| ✓ | BbhLogicalDeductionFive207BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT869 |
| ✓ | BbhLogicalDeductionFive208BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT1 595 |
| ✓ | BbhLogicalDeductionFive209BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT2 229 |
| ✓ | BbhLogicalDeductionFive210BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 390 |
| ✓ | BbhLogicalDeductionFive211BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 744 |
| ✓ | BbhLogicalDeductionFive212BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 866 |
| ✓ | BbhLogicalDeductionFive213BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 309 |
| ✓ | BbhLogicalDeductionFive214BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 417 |
| ✓ | BbhLogicalDeductionFive215BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 780 |
| ✓ | BbhLogicalDeductionFive216BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 798 |
| ✓ | BbhLogicalDeductionFive217BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 866 |
| ✓ | BbhLogicalDeductionFive218BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 120 |
| ✓ | BbhLogicalDeductionFive219BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 879 |
| ✓ | BbhLogicalDeductionFive220BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 094 |
Reasoning - BBH logical deduction - order 5 objects (set 3)
29 passed |
100.0% |
IN31 705 OUT42 589 | |
| ✓ | BbhLogicalDeductionFive221BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 278 |
| ✓ | BbhLogicalDeductionFive222BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT2 215 |
| ✓ | BbhLogicalDeductionFive223BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 422 |
| ✓ | BbhLogicalDeductionFive224BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 537 |
| ✓ | BbhLogicalDeductionFive225BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 695 |
| ✓ | BbhLogicalDeductionFive226BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 413 |
| ✓ | BbhLogicalDeductionFive227BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 857 |
| ✓ | BbhLogicalDeductionFive228BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 784 |
| ✓ | BbhLogicalDeductionFive229BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT818 |
| ✓ | BbhLogicalDeductionFive230BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 404 |
| ✓ | BbhLogicalDeductionFive231BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT2 106 |
| ✓ | BbhLogicalDeductionFive232BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT777 |
| ✓ | BbhLogicalDeductionFive233BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT2 242 |
| ✓ | BbhLogicalDeductionFive234BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 510 |
| ✓ | BbhLogicalDeductionFive235BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 566 |
| ✓ | BbhLogicalDeductionFive236BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 126 |
| ✓ | BbhLogicalDeductionFive237BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 258 |
| ✓ | BbhLogicalDeductionFive238BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 864 |
| ✓ | BbhLogicalDeductionFive239BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 383 |
| ✓ | BbhLogicalDeductionFive240BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 363 |
| ✓ | BbhLogicalDeductionFive241BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT2 133 |
| ✓ | BbhLogicalDeductionFive242BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT732 |
| ✓ | BbhLogicalDeductionFive243BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 333 |
| ✓ | BbhLogicalDeductionFive244BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 907 |
| ✓ | BbhLogicalDeductionFive245BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT1 002 |
| ✓ | BbhLogicalDeductionFive246BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT1 368 |
| ✓ | BbhLogicalDeductionFive247BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 332 |
| ✓ | BbhLogicalDeductionFive248BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT905 |
| ✓ | BbhLogicalDeductionFive249BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 259 |
Reasoning - BBH logical deduction - order 5 objects (set 4)
32 passed |
100.0% |
IN35 046 OUT48 304 | |
| ✓ | BbhLogicalDeductionFiveObjects31BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT854 |
| ✓ | BbhLogicalDeductionFiveObjects32BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 308 |
| ✓ | BbhLogicalDeductionFiveObjects33BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 563 |
| ✓ | BbhLogicalDeductionFiveObjects34BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 214 |
| ✓ | BbhLogicalDeductionFiveObjects35BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 424 |
| ✓ | BbhLogicalDeductionFiveObjects36BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT847 |
| ✓ | BbhLogicalDeductionFiveObjects37BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 468 |
| ✓ | BbhLogicalDeductionFiveObjects38BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT1 432 |
| ✓ | BbhLogicalDeductionFiveObjects39BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 617 |
| ✓ | BbhLogicalDeductionFiveObjects40BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT1 362 |
| ✓ | BbhLogicalDeductionFiveObjects41BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT2 554 |
| ✓ | BbhLogicalDeductionFiveObjects42BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT724 |
| ✓ | BbhLogicalDeductionFiveObjects43BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 646 |
| ✓ | BbhLogicalDeductionFiveObjects44BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 232 |
| ✓ | BbhLogicalDeductionFiveObjects45BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 891 |
| ✓ | BbhLogicalDeductionFiveObjects46BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 474 |
| ✓ | BbhLogicalDeductionFiveObjects47BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 724 |
| ✓ | BbhLogicalDeductionFiveObjects48BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 048 |
| ✓ | BbhLogicalDeductionFiveObjects49BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 207 |
| ✓ | BbhLogicalDeductionFiveObjects50BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 115 |
| ✓ | BbhLogicalDeductionFiveObjects51BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 396 |
| ✓ | BbhLogicalDeductionFiveObjects52BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 138 |
| ✓ | BbhLogicalDeductionFiveObjects53BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT3 352 |
| ✓ | BbhLogicalDeductionFiveObjects54BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 964 |
| ✓ | BbhLogicalDeductionFiveObjects55BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 873 |
| ✓ | BbhLogicalDeductionFiveObjects56BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 052 |
| ✓ | BbhLogicalDeductionFiveObjects57BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT2 183 |
| ✓ | BbhLogicalDeductionFiveObjects58BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 052 |
| ✓ | BbhLogicalDeductionFiveObjects59BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT2 265 |
| ✓ | BbhLogicalDeductionFiveObjects60BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 173 |
| ✓ | BbhLogicalDeductionFiveObjects61BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 305 |
| ✓ | BbhLogicalDeductionFiveObjects62BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 847 |
Reasoning - BBH logical deduction - order 5 objects (set 5)
32 passed |
100.0% |
IN35 054 OUT52 949 | |
| ✓ | BbhLogicalDeductionFiveObjects63BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 173 |
| ✓ | BbhLogicalDeductionFiveObjects64BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 674 |
| ✓ | BbhLogicalDeductionFiveObjects65BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT4 369 |
| ✓ | BbhLogicalDeductionFiveObjects66BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT2 163 |
| ✓ | BbhLogicalDeductionFiveObjects67BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 474 |
| ✓ | BbhLogicalDeductionFiveObjects68BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 333 |
| ✓ | BbhLogicalDeductionFiveObjects69BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 517 |
| ✓ | BbhLogicalDeductionFiveObjects70BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 475 |
| ✓ | BbhLogicalDeductionFiveObjects71BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT2 216 |
| ✓ | BbhLogicalDeductionFiveObjects72BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 335 |
| ✓ | BbhLogicalDeductionFiveObjects73BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 248 |
| ✓ | BbhLogicalDeductionFiveObjects74BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT828 |
| ✓ | BbhLogicalDeductionFiveObjects75BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 201 |
| ✓ | BbhLogicalDeductionFiveObjects76BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 133 |
| ✓ | BbhLogicalDeductionFiveObjects77BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 868 |
| ✓ | BbhLogicalDeductionFiveObjects78BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 571 |
| ✓ | BbhLogicalDeductionFiveObjects79BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 043 |
| ✓ | BbhLogicalDeductionFiveObjects80BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 551 |
| ✓ | BbhLogicalDeductionFiveObjects81BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 295 |
| ✓ | BbhLogicalDeductionFiveObjects82BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT2 303 |
| ✓ | BbhLogicalDeductionFiveObjects83BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 692 |
| ✓ | BbhLogicalDeductionFiveObjects84BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 480 |
| ✓ | BbhLogicalDeductionFiveObjects85BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT1 577 |
| ✓ | BbhLogicalDeductionFiveObjects86BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 240 |
| ✓ | BbhLogicalDeductionFiveObjects87BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 876 |
| ✓ | BbhLogicalDeductionFiveObjects88BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT1 685 |
| ✓ | BbhLogicalDeductionFiveObjects89BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 178 |
| ✓ | BbhLogicalDeductionFiveObjects90BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT2 083 |
| ✓ | BbhLogicalDeductionFiveObjects91BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 995 |
| ✓ | BbhLogicalDeductionFiveObjects92BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT1 131 |
| ✓ | BbhLogicalDeductionFiveObjects93BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 809 |
| ✓ | BbhLogicalDeductionFiveObjects94BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT1 433 |
Reasoning - BBH logical deduction - order 5 objects (set 6)
32 passed |
100.0% |
IN34 988 OUT46 932 | |
| ✓ | BbhLogicalDeductionFiveObjects100BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 344 |
| ✓ | BbhLogicalDeductionFiveObjects101BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 656 |
| ✓ | BbhLogicalDeductionFiveObjects102BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT1 801 |
| ✓ | BbhLogicalDeductionFiveObjects103BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT2 015 |
| ✓ | BbhLogicalDeductionFiveObjects104BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT842 |
| ✓ | BbhLogicalDeductionFiveObjects105BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 677 |
| ✓ | BbhLogicalDeductionFiveObjects106BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 677 |
| ✓ | BbhLogicalDeductionFiveObjects107BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 167 |
| ✓ | BbhLogicalDeductionFiveObjects108BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 505 |
| ✓ | BbhLogicalDeductionFiveObjects109BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 215 |
| ✓ | BbhLogicalDeductionFiveObjects110BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 936 |
| ✓ | BbhLogicalDeductionFiveObjects111BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 240 |
| ✓ | BbhLogicalDeductionFiveObjects112BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 937 |
| ✓ | BbhLogicalDeductionFiveObjects113BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 271 |
| ✓ | BbhLogicalDeductionFiveObjects114BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 036 |
| ✓ | BbhLogicalDeductionFiveObjects115BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 272 |
| ✓ | BbhLogicalDeductionFiveObjects116BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT1 759 |
| ✓ | BbhLogicalDeductionFiveObjects117BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 377 |
| ✓ | BbhLogicalDeductionFiveObjects118BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 358 |
| ✓ | BbhLogicalDeductionFiveObjects119BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT2 116 |
| ✓ | BbhLogicalDeductionFiveObjects120BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 363 |
| ✓ | BbhLogicalDeductionFiveObjects121BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT1 581 |
| ✓ | BbhLogicalDeductionFiveObjects122BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 206 |
| ✓ | BbhLogicalDeductionFiveObjects123BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 993 |
| ✓ | BbhLogicalDeductionFiveObjects124BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT1 870 |
| ✓ | BbhLogicalDeductionFiveObjects125BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT923 |
| ✓ | BbhLogicalDeductionFiveObjects126BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 609 |
| ✓ | BbhLogicalDeductionFiveObjects95BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 604 |
| ✓ | BbhLogicalDeductionFiveObjects96BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 198 |
| ✓ | BbhLogicalDeductionFiveObjects97BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 224 |
| ✓ | BbhLogicalDeductionFiveObjects98BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT1 010 |
| ✓ | BbhLogicalDeductionFiveObjects99BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 150 |
Reasoning - BBH logical deduction - order 5 objects (set 7)
32 passed |
100.0% |
IN35 021 OUT47 904 | |
| ✓ | BbhLogicalDeductionFiveObjects127BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 031 |
| ✓ | BbhLogicalDeductionFiveObjects128BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 876 |
| ✓ | BbhLogicalDeductionFiveObjects129BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT905 |
| ✓ | BbhLogicalDeductionFiveObjects130BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 937 |
| ✓ | BbhLogicalDeductionFiveObjects131BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT2 058 |
| ✓ | BbhLogicalDeductionFiveObjects132BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 797 |
| ✓ | BbhLogicalDeductionFiveObjects133BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 622 |
| ✓ | BbhLogicalDeductionFiveObjects134BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT1 119 |
| ✓ | BbhLogicalDeductionFiveObjects135BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 980 |
| ✓ | BbhLogicalDeductionFiveObjects136BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT2 342 |
| ✓ | BbhLogicalDeductionFiveObjects137BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 758 |
| ✓ | BbhLogicalDeductionFiveObjects138BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 805 |
| ✓ | BbhLogicalDeductionFiveObjects139BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 217 |
| ✓ | BbhLogicalDeductionFiveObjects140BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 644 |
| ✓ | BbhLogicalDeductionFiveObjects141BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 293 |
| ✓ | BbhLogicalDeductionFiveObjects142BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 471 |
| ✓ | BbhLogicalDeductionFiveObjects143BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT865 |
| ✓ | BbhLogicalDeductionFiveObjects144BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 611 |
| ✓ | BbhLogicalDeductionFiveObjects145BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 664 |
| ✓ | BbhLogicalDeductionFiveObjects146BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 004 |
| ✓ | BbhLogicalDeductionFiveObjects147BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 220 |
| ✓ | BbhLogicalDeductionFiveObjects148BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT465 |
| ✓ | BbhLogicalDeductionFiveObjects149BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 407 |
| ✓ | BbhLogicalDeductionFiveObjects150BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT2 027 |
| ✓ | BbhLogicalDeductionFiveObjects151BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 370 |
| ✓ | BbhLogicalDeductionFiveObjects152BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT1 436 |
| ✓ | BbhLogicalDeductionFiveObjects153BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT1 199 |
| ✓ | BbhLogicalDeductionFiveObjects154BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 220 |
| ✓ | BbhLogicalDeductionFiveObjects155BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 225 |
| ✓ | BbhLogicalDeductionFiveObjects156BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT1 408 |
| ✓ | BbhLogicalDeductionFiveObjects157BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 512 |
| ✓ | BbhLogicalDeductionFiveObjects158BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 416 |
Reasoning - BBH logical deduction - order 5 objects (set 8)
30 passed |
100.0% |
IN32 715 OUT45 468 | |
| ✓ | BbhLogicalDeductionFiveObjects159BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 169 |
| ✓ | BbhLogicalDeductionFiveObjects160BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 442 |
| ✓ | BbhLogicalDeductionFiveObjects161BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT2 128 |
| ✓ | BbhLogicalDeductionFiveObjects162BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 279 |
| ✓ | BbhLogicalDeductionFiveObjects163BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 549 |
| ✓ | BbhLogicalDeductionFiveObjects164BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 759 |
| ✓ | BbhLogicalDeductionFiveObjects165BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT2 229 |
| ✓ | BbhLogicalDeductionFiveObjects166BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT774 |
| ✓ | BbhLogicalDeductionFiveObjects167BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 226 |
| ✓ | BbhLogicalDeductionFiveObjects168BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 115 |
| ✓ | BbhLogicalDeductionFiveObjects169BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 250 |
| ✓ | BbhLogicalDeductionFiveObjects170BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 966 |
| ✓ | BbhLogicalDeductionFiveObjects171BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT2 440 |
| ✓ | BbhLogicalDeductionFiveObjects172BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 886 |
| ✓ | BbhLogicalDeductionFiveObjects173BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 235 |
| ✓ | BbhLogicalDeductionFiveObjects174BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT811 |
| ✓ | BbhLogicalDeductionFiveObjects175BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 557 |
| ✓ | BbhLogicalDeductionFiveObjects176BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT1 731 |
| ✓ | BbhLogicalDeductionFiveObjects177BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 745 |
| ✓ | BbhLogicalDeductionFiveObjects178BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 060 |
| ✓ | BbhLogicalDeductionFiveObjects179BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 479 |
| ✓ | BbhLogicalDeductionFiveObjects180BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT1 839 |
| ✓ | BbhLogicalDeductionFiveObjects181BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 367 |
| ✓ | BbhLogicalDeductionFiveObjects182BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 433 |
| ✓ | BbhLogicalDeductionFiveObjects183BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 512 |
| ✓ | BbhLogicalDeductionFiveObjects184BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT2 105 |
| ✓ | BbhLogicalDeductionFiveObjects185BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 459 |
| ✓ | BbhLogicalDeductionFiveObjects186BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT831 |
| ✓ | BbhLogicalDeductionFiveObjects187BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT790 |
| ✓ | BbhLogicalDeductionFiveObjects188BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 302 |
Reasoning - BBH logical deduction - order 7 objects (set 1)
32 passed1 failed |
97.0% |
IN37 680 OUT74 172 | |
| ✓ | BbhLogicalDeductionSeven10BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 101 |
| ✓ | BbhLogicalDeductionSeven11BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 389 |
| ✓ | BbhLogicalDeductionSeven12BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT2 375 |
| ✓ | BbhLogicalDeductionSeven13BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 565 |
| ✓ | BbhLogicalDeductionSeven14BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT1 597 |
| ✓ | BbhLogicalDeductionSeven15BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT3 054 |
| ✓ | BbhLogicalDeductionSeven16BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT5 024 |
| ✓ | BbhLogicalDeductionSeven17BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 884 |
| ✓ | BbhLogicalDeductionSeven18BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT3 262 |
| ✓ | BbhLogicalDeductionSeven19BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 615 |
| ✓ | BbhLogicalDeductionSeven20BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT2 467 |
| ✓ | BbhLogicalDeductionSeven21BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT2 647 |
| ✓ | BbhLogicalDeductionSeven22BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT2 000 |
| ✓ | BbhLogicalDeductionSeven23BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT1 976 |
| ✓ | BbhLogicalDeductionSeven24BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 609 |
| ✓ | BbhLogicalDeductionSeven25BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 630 |
| ✓ | BbhLogicalDeductionSeven26BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT6 237 |
| ✓ | BbhLogicalDeductionSeven27BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 213 |
| ✓ | BbhLogicalDeductionSeven28BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 069 |
| ✓ | BbhLogicalDeductionSeven29BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT2 229 |
| ✓ | BbhLogicalDeductionSeven30BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 141 OUT2 305 |
| ✓ | BbhLogicalDeductionSeven31BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 082 OUT2 084 |
| ✓ | BbhLogicalDeductionSeven32BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 078 OUT1 670 |
| ✓ | BbhLogicalDeductionSeven33BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 075 OUT1 498 |
| ✓ | BbhLogicalDeductionSeven34BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT2 259 |
| ✓ | BbhLogicalDeductionSeven35BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT2 439 |
| ✓ | BbhLogicalDeductionSeven36BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT1 866 |
| × | BbhLogicalDeductionSeven37BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhLogicalDeductionSeven37BQ: expected (C), got [(E)] |
Failed | IN1 151 OUT1 735 |
| ✓ | BbhLogicalDeductionSeven38BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 132 OUT1 268 |
| ✓ | BbhLogicalDeductionSeven39BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 098 OUT1 808 |
| ✓ | BbhLogicalDeductionSeven40BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 024 |
| ✓ | BbhLogicalDeductionSeven8BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 134 OUT1 734 |
| ✓ | BbhLogicalDeductionSeven9BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 539 |
Reasoning - BBH logical deduction - order 7 objects (set 2)
33 passed |
100.0% |
IN37 367 OUT67 170 | |
| ✓ | BbhLogicalDeductionSeven41BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 264 |
| ✓ | BbhLogicalDeductionSeven42BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 133 OUT2 070 |
| ✓ | BbhLogicalDeductionSeven43BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT1 158 |
| ✓ | BbhLogicalDeductionSeven44BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 131 OUT1 960 |
| ✓ | BbhLogicalDeductionSeven45BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 673 |
| ✓ | BbhLogicalDeductionSeven46BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT1 253 |
| ✓ | BbhLogicalDeductionSeven47BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT3 313 |
| ✓ | BbhLogicalDeductionSeven48BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 335 |
| ✓ | BbhLogicalDeductionSeven49BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 137 OUT1 576 |
| ✓ | BbhLogicalDeductionSeven50BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 618 |
| ✓ | BbhLogicalDeductionSeven51BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 695 |
| ✓ | BbhLogicalDeductionSeven52BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 091 OUT1 749 |
| ✓ | BbhLogicalDeductionSeven53BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT2 030 |
| ✓ | BbhLogicalDeductionSeven54BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 078 OUT2 254 |
| ✓ | BbhLogicalDeductionSeven55BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 307 |
| ✓ | BbhLogicalDeductionSeven56BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT2 418 |
| ✓ | BbhLogicalDeductionSeven57BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT2 445 |
| ✓ | BbhLogicalDeductionSeven58BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 594 |
| ✓ | BbhLogicalDeductionSeven59BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT2 308 |
| ✓ | BbhLogicalDeductionSeven60BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 091 OUT2 512 |
| ✓ | BbhLogicalDeductionSeven61BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 448 |
| ✓ | BbhLogicalDeductionSeven62BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 437 |
| ✓ | BbhLogicalDeductionSeven63BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 655 |
| ✓ | BbhLogicalDeductionSeven64BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 082 OUT2 046 |
| ✓ | BbhLogicalDeductionSeven65BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 694 |
| ✓ | BbhLogicalDeductionSeven66BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT2 093 |
| ✓ | BbhLogicalDeductionSeven67BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT2 163 |
| ✓ | BbhLogicalDeductionSeven68BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 137 OUT1 918 |
| ✓ | BbhLogicalDeductionSeven69BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 750 |
| ✓ | BbhLogicalDeductionSeven70BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT3 198 |
| ✓ | BbhLogicalDeductionSeven71BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT2 096 |
| ✓ | BbhLogicalDeductionSeven72BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 133 OUT2 013 |
| ✓ | BbhLogicalDeductionSeven73BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 141 OUT2 127 |
Reasoning - BBH logical deduction - order 7 objects (set 3)
31 passed |
100.0% |
IN35 350 OUT63 528 | |
| ✓ | BbhLogicalDeductionSeven100BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 137 OUT2 612 |
| ✓ | BbhLogicalDeductionSeven101BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT1 428 |
| ✓ | BbhLogicalDeductionSeven102BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT2 153 |
| ✓ | BbhLogicalDeductionSeven103BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT1 221 |
| ✓ | BbhLogicalDeductionSeven104BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 582 |
| ✓ | BbhLogicalDeductionSeven74BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 142 OUT2 354 |
| ✓ | BbhLogicalDeductionSeven75BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT3 186 |
| ✓ | BbhLogicalDeductionSeven76BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 968 |
| ✓ | BbhLogicalDeductionSeven77BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT2 171 |
| ✓ | BbhLogicalDeductionSeven78BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT2 244 |
| ✓ | BbhLogicalDeductionSeven79BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 086 |
| ✓ | BbhLogicalDeductionSeven80BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT1 514 |
| ✓ | BbhLogicalDeductionSeven81BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 181 OUT3 344 |
| ✓ | BbhLogicalDeductionSeven82BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 597 |
| ✓ | BbhLogicalDeductionSeven83BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 136 OUT2 747 |
| ✓ | BbhLogicalDeductionSeven84BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT1 616 |
| ✓ | BbhLogicalDeductionSeven85BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 141 OUT1 966 |
| ✓ | BbhLogicalDeductionSeven86BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 593 |
| ✓ | BbhLogicalDeductionSeven87BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 141 OUT2 117 |
| ✓ | BbhLogicalDeductionSeven88BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT1 195 |
| ✓ | BbhLogicalDeductionSeven89BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 133 OUT1 833 |
| ✓ | BbhLogicalDeductionSeven90BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 078 OUT1 408 |
| ✓ | BbhLogicalDeductionSeven91BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 576 |
| ✓ | BbhLogicalDeductionSeven92BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 510 |
| ✓ | BbhLogicalDeductionSeven93BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT2 209 |
| ✓ | BbhLogicalDeductionSeven94BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT2 017 |
| ✓ | BbhLogicalDeductionSeven95BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT2 201 |
| ✓ | BbhLogicalDeductionSeven96BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 830 |
| ✓ | BbhLogicalDeductionSeven97BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT2 197 |
| ✓ | BbhLogicalDeductionSeven98BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT2 536 |
| ✓ | BbhLogicalDeductionSeven99BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT2 517 |
Reasoning - BBH logical deduction - order 7 objects (set 4)
32 passed |
100.0% |
IN36 292 OUT67 473 | |
| ✓ | BbhLogicalDeductionSeven105BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT2 329 |
| ✓ | BbhLogicalDeductionSeven106BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT2 655 |
| ✓ | BbhLogicalDeductionSeven107BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT2 101 |
| ✓ | BbhLogicalDeductionSeven108BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 078 OUT1 922 |
| ✓ | BbhLogicalDeductionSeven109BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT4 851 |
| ✓ | BbhLogicalDeductionSeven110BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT2 748 |
| ✓ | BbhLogicalDeductionSeven111BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT1 884 |
| ✓ | BbhLogicalDeductionSeven112BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 099 OUT1 420 |
| ✓ | BbhLogicalDeductionSeven113BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT1 798 |
| ✓ | BbhLogicalDeductionSeven114BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT2 312 |
| ✓ | BbhLogicalDeductionSeven115BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT2 615 |
| ✓ | BbhLogicalDeductionSeven116BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 800 |
| ✓ | BbhLogicalDeductionSeven117BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 768 |
| ✓ | BbhLogicalDeductionSeven118BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT2 230 |
| ✓ | BbhLogicalDeductionSeven119BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 791 |
| ✓ | BbhLogicalDeductionSeven120BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT1 301 |
| ✓ | BbhLogicalDeductionSeven121BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 096 OUT2 517 |
| ✓ | BbhLogicalDeductionSeven122BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT1 622 |
| ✓ | BbhLogicalDeductionSeven123BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 092 OUT1 355 |
| ✓ | BbhLogicalDeductionSeven124BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 078 OUT1 644 |
| ✓ | BbhLogicalDeductionSeven125BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT1 898 |
| ✓ | BbhLogicalDeductionSeven126BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 359 |
| ✓ | BbhLogicalDeductionSeven127BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 431 |
| ✓ | BbhLogicalDeductionSeven128BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 841 |
| ✓ | BbhLogicalDeductionSeven129BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 941 |
| ✓ | BbhLogicalDeductionSeven130BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 088 OUT2 393 |
| ✓ | BbhLogicalDeductionSeven131BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT2 046 |
| ✓ | BbhLogicalDeductionSeven132BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 860 |
| ✓ | BbhLogicalDeductionSeven133BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 891 |
| ✓ | BbhLogicalDeductionSeven134BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 623 |
| ✓ | BbhLogicalDeductionSeven135BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 134 OUT2 352 |
| ✓ | BbhLogicalDeductionSeven136BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT2 175 |
Reasoning - BBH logical deduction - order 7 objects (set 5)
32 passed |
100.0% |
IN36 403 OUT64 911 | |
| ✓ | BbhLogicalDeductionSeven137BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 431 |
| ✓ | BbhLogicalDeductionSeven138BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT2 785 |
| ✓ | BbhLogicalDeductionSeven139BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 809 |
| ✓ | BbhLogicalDeductionSeven140BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 523 |
| ✓ | BbhLogicalDeductionSeven141BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 110 |
| ✓ | BbhLogicalDeductionSeven142BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT2 314 |
| ✓ | BbhLogicalDeductionSeven143BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT2 226 |
| ✓ | BbhLogicalDeductionSeven144BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT2 397 |
| ✓ | BbhLogicalDeductionSeven145BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 997 |
| ✓ | BbhLogicalDeductionSeven146BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT3 867 |
| ✓ | BbhLogicalDeductionSeven147BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 555 |
| ✓ | BbhLogicalDeductionSeven148BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT2 198 |
| ✓ | BbhLogicalDeductionSeven149BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT3 006 |
| ✓ | BbhLogicalDeductionSeven150BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT2 059 |
| ✓ | BbhLogicalDeductionSeven151BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 145 OUT1 568 |
| ✓ | BbhLogicalDeductionSeven152BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT2 104 |
| ✓ | BbhLogicalDeductionSeven153BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 092 OUT1 665 |
| ✓ | BbhLogicalDeductionSeven154BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 903 |
| ✓ | BbhLogicalDeductionSeven155BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 154 OUT1 642 |
| ✓ | BbhLogicalDeductionSeven156BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 174 |
| ✓ | BbhLogicalDeductionSeven157BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 082 OUT1 894 |
| ✓ | BbhLogicalDeductionSeven158BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 735 |
| ✓ | BbhLogicalDeductionSeven159BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 711 |
| ✓ | BbhLogicalDeductionSeven160BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 103 |
| ✓ | BbhLogicalDeductionSeven161BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 088 OUT1 598 |
| ✓ | BbhLogicalDeductionSeven162BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT3 050 |
| ✓ | BbhLogicalDeductionSeven163BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT1 501 |
| ✓ | BbhLogicalDeductionSeven164BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT2 159 |
| ✓ | BbhLogicalDeductionSeven165BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 564 |
| ✓ | BbhLogicalDeductionSeven166BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 526 |
| ✓ | BbhLogicalDeductionSeven167BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 089 OUT1 731 |
| ✓ | BbhLogicalDeductionSeven168BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 006 |
Reasoning - BBH logical deduction - order 7 objects (set 6)
28 passed4 failed |
87.5% |
IN36 240 OUT67 299 | |
| ✓ | BbhLogicalDeductionSeven169BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 470 |
| ✓ | BbhLogicalDeductionSeven170BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT1 420 |
| ✓ | BbhLogicalDeductionSeven171BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 235 |
| ✓ | BbhLogicalDeductionSeven172BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 085 OUT1 643 |
| ✓ | BbhLogicalDeductionSeven173BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT3 789 |
| ✓ | BbhLogicalDeductionSeven174BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT2 103 |
| ✓ | BbhLogicalDeductionSeven175BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 483 |
| × | BbhLogicalDeductionSeven176BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: got == Norm("(F)")
Context: BbhLogicalDeductionSeven176BR: expected (F), got [a]
Values:
got = "a" |
Failed | IN1 144 OUT6 782 |
| ✓ | BbhLogicalDeductionSeven177BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT2 572 |
| ✓ | BbhLogicalDeductionSeven178BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 640 |
| ✓ | BbhLogicalDeductionSeven179BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 524 |
| ✓ | BbhLogicalDeductionSeven180BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 145 OUT1 727 |
| ✓ | BbhLogicalDeductionSeven181BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 093 OUT1 984 |
| ✓ | BbhLogicalDeductionSeven182BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 949 |
| ✓ | BbhLogicalDeductionSeven183BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 075 OUT1 794 |
| × | BbhLogicalDeductionSeven184BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhLogicalDeductionSeven184BR: expected (D), got [b]
Values:
got = "b" |
Failed | IN1 169 OUT5 033 |
| ✓ | BbhLogicalDeductionSeven185BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 854 |
| ✓ | BbhLogicalDeductionSeven186BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 095 OUT1 123 |
| × | BbhLogicalDeductionSeven187BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhLogicalDeductionSeven187BR: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 157 OUT2 043 |
| ✓ | BbhLogicalDeductionSeven188BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 095 OUT1 097 |
| ✓ | BbhLogicalDeductionSeven189BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 175 OUT1 940 |
| ✓ | BbhLogicalDeductionSeven190BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 569 |
| ✓ | BbhLogicalDeductionSeven191BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 141 OUT1 976 |
| ✓ | BbhLogicalDeductionSeven192BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 696 |
| × | BbhLogicalDeductionSeven193BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhLogicalDeductionSeven193BR: expected (C), got [b]
Values:
got = "b" |
Failed | IN1 156 OUT2 715 |
| ✓ | BbhLogicalDeductionSeven194BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 681 |
| ✓ | BbhLogicalDeductionSeven195BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 141 OUT1 629 |
| ✓ | BbhLogicalDeductionSeven196BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT1 361 |
| ✓ | BbhLogicalDeductionSeven197BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 132 OUT1 641 |
| ✓ | BbhLogicalDeductionSeven198BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 823 |
| ✓ | BbhLogicalDeductionSeven199BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 481 |
| ✓ | BbhLogicalDeductionSeven200BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 122 OUT1 522 |
Reasoning - BBH logical deduction - order 7 objects (set 7)
32 passed |
100.0% |
IN36 365 OUT64 566 | |
| ✓ | BbhLogicalDeductionSeven201BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 141 OUT1 459 |
| ✓ | BbhLogicalDeductionSeven202BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 955 |
| ✓ | BbhLogicalDeductionSeven203BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 142 OUT1 360 |
| ✓ | BbhLogicalDeductionSeven204BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT2 784 |
| ✓ | BbhLogicalDeductionSeven205BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 377 |
| ✓ | BbhLogicalDeductionSeven206BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 138 OUT1 756 |
| ✓ | BbhLogicalDeductionSeven207BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 155 |
| ✓ | BbhLogicalDeductionSeven208BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 091 OUT1 634 |
| ✓ | BbhLogicalDeductionSeven209BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 092 OUT2 049 |
| ✓ | BbhLogicalDeductionSeven210BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 663 |
| ✓ | BbhLogicalDeductionSeven211BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 736 |
| ✓ | BbhLogicalDeductionSeven212BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 724 |
| ✓ | BbhLogicalDeductionSeven213BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 382 |
| ✓ | BbhLogicalDeductionSeven214BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT1 539 |
| ✓ | BbhLogicalDeductionSeven215BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 137 OUT1 889 |
| ✓ | BbhLogicalDeductionSeven216BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 145 OUT2 048 |
| ✓ | BbhLogicalDeductionSeven217BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 122 OUT1 731 |
| ✓ | BbhLogicalDeductionSeven218BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 137 OUT2 209 |
| ✓ | BbhLogicalDeductionSeven219BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT1 993 |
| ✓ | BbhLogicalDeductionSeven220BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 555 |
| ✓ | BbhLogicalDeductionSeven221BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 088 OUT2 115 |
| ✓ | BbhLogicalDeductionSeven222BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 137 OUT2 413 |
| ✓ | BbhLogicalDeductionSeven223BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT2 770 |
| ✓ | BbhLogicalDeductionSeven224BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 095 OUT1 326 |
| ✓ | BbhLogicalDeductionSeven225BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 181 OUT3 190 |
| ✓ | BbhLogicalDeductionSeven226BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 475 |
| ✓ | BbhLogicalDeductionSeven227BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT1 756 |
| ✓ | BbhLogicalDeductionSeven228BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT2 080 |
| ✓ | BbhLogicalDeductionSeven229BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 556 |
| ✓ | BbhLogicalDeductionSeven230BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 140 OUT2 626 |
| ✓ | BbhLogicalDeductionSeven231BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT2 471 |
| ✓ | BbhLogicalDeductionSeven232BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 790 |
Reasoning - BBH logical deduction - order 7 objects (set 8)
30 passed |
100.0% |
IN33 128 OUT46 980 | |
| ✓ | BbhLogicalDeductionSeven233BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT2 380 |
| ✓ | BbhLogicalDeductionSeven234BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 132 OUT1 963 |
| ✓ | BbhLogicalDeductionSeven235BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 078 OUT1 451 |
| ✓ | BbhLogicalDeductionSeven236BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 154 OUT1 983 |
| ✓ | BbhLogicalDeductionSeven237BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 713 |
| ✓ | BbhLogicalDeductionSeven238BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 441 |
| ✓ | BbhLogicalDeductionSeven239BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 946 |
| ✓ | BbhLogicalDeductionSeven240BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 166 |
| ✓ | BbhLogicalDeductionSeven241BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT1 493 |
| ✓ | BbhLogicalDeductionSeven242BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 175 OUT1 880 |
| ✓ | BbhLogicalDeductionSeven243BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 614 |
| ✓ | BbhLogicalDeductionSeven244BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT2 153 |
| ✓ | BbhLogicalDeductionSeven245BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 095 OUT1 956 |
| ✓ | BbhLogicalDeductionSeven246BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 150 OUT1 491 |
| ✓ | BbhLogicalDeductionSeven247BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT2 307 |
| ✓ | BbhLogicalDeductionSeven248BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT2 038 |
| ✓ | BbhLogicalDeductionSeven249BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 136 OUT2 124 |
| ✓ | BbhLogicalDeductionThree10BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT293 |
| ✓ | BbhLogicalDeductionThree11BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT565 |
| ✓ | BbhLogicalDeductionThree12BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 214 |
| ✓ | BbhLogicalDeductionThree13BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT982 |
| ✓ | BbhLogicalDeductionThree14BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 644 |
| ✓ | BbhLogicalDeductionThree15BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT694 |
| ✓ | BbhLogicalDeductionThree16BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT1 066 |
| ✓ | BbhLogicalDeductionThree17BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT969 |
| ✓ | BbhLogicalDeductionThree18BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT778 |
| ✓ | BbhLogicalDeductionThree19BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT780 |
| ✓ | BbhLogicalDeductionThree20BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT885 |
| ✓ | BbhLogicalDeductionThree8BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 322 |
| ✓ | BbhLogicalDeductionThree9BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT689 |
Reasoning - BBH movie recommendation (set 1)
30 passed2 failed |
93.8% |
IN32 509 OUT50 879 | |
| ✓ | BbhLogicalDeductionThreeObjects243BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT930 |
| ✓ | BbhLogicalDeductionThreeObjects244BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT1 097 |
| ✓ | BbhLogicalDeductionThreeObjects245BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT1 707 |
| ✓ | BbhLogicalDeductionThreeObjects246BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT1 075 |
| ✓ | BbhLogicalDeductionThreeObjects247BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT881 |
| ✓ | BbhLogicalDeductionThreeObjects248BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 031 |
| ✓ | BbhLogicalDeductionThreeObjects249BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT732 |
| ✓ | BbhMovieRecommendation10BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 573 |
| ✓ | BbhMovieRecommendation11BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 613 |
| ✓ | BbhMovieRecommendation12BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 447 |
| ✓ | BbhMovieRecommendation13BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT2 066 |
| ✓ | BbhMovieRecommendation14BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 229 |
| ✓ | BbhMovieRecommendation15BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 606 |
| ✓ | BbhMovieRecommendation16BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 586 |
| ✓ | BbhMovieRecommendation17BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 572 |
| ✓ | BbhMovieRecommendation18BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 230 |
| ✓ | BbhMovieRecommendation19BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT2 481 |
| ✓ | BbhMovieRecommendation20BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 664 |
| ✓ | BbhMovieRecommendation21BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT3 586 |
| ✓ | BbhMovieRecommendation22BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT1 266 |
| ✓ | BbhMovieRecommendation23BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 281 |
| ✓ | BbhMovieRecommendation24BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 487 |
| × | BbhMovieRecommendation25BTFind a movie similar to Army of Darkness, Go, The Sixth Sense, Alien:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhMovieRecommendation25BT: expected (A), got [(B)] |
Failed | IN998 OUT2 599 |
| ✓ | BbhMovieRecommendation26BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 421 |
| ✓ | BbhMovieRecommendation27BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 206 |
| ✓ | BbhMovieRecommendation28BTFind a movie similar to Toy Story, The Mask, Batman, The Lion King:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 909 |
| ✓ | BbhMovieRecommendation29BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 221 |
| ✓ | BbhMovieRecommendation30BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT2 463 |
| ✓ | BbhMovieRecommendation31BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 854 |
| ✓ | BbhMovieRecommendation32BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 152 |
| ✓ | BbhMovieRecommendation8BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 444 |
| × | BbhMovieRecommendation9BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhMovieRecommendation9BT: expected (C), got [(B)] |
Failed | IN1 021 OUT1 470 |
Reasoning - BBH movie recommendation (set 2)
29 passed3 failed |
90.6% |
IN32 260 OUT63 249 | |
| ✓ | BbhMovieRecommendation33BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT2 115 |
| ✓ | BbhMovieRecommendation34BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT2 020 |
| ✓ | BbhMovieRecommendation35BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT2 393 |
| ✓ | BbhMovieRecommendation36BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 367 |
| ✓ | BbhMovieRecommendation37BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 797 |
| ✓ | BbhMovieRecommendation38BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 705 |
| ✓ | BbhMovieRecommendation39BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT2 844 |
| ✓ | BbhMovieRecommendation40BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 944 |
| ✓ | BbhMovieRecommendation41BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 418 |
| ✓ | BbhMovieRecommendation42BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 055 |
| ✓ | BbhMovieRecommendation43BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 219 |
| ✓ | BbhMovieRecommendation44BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 942 |
| ✓ | BbhMovieRecommendation45BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 345 |
| ✓ | BbhMovieRecommendation46BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT991 |
| ✓ | BbhMovieRecommendation47BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 254 |
| ✓ | BbhMovieRecommendation48BTFind a movie similar to Batman, The Mask, The Lion King, Pretty Woman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 689 |
| ✓ | BbhMovieRecommendation49BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 114 |
| ✓ | BbhMovieRecommendation50BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT1 705 |
| ✓ | BbhMovieRecommendation51BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 641 |
| × | BbhMovieRecommendation52BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhMovieRecommendation52BT: expected (A), got [(B)] |
Failed | IN1 000 OUT2 684 |
| ✓ | BbhMovieRecommendation53BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 523 |
| ✓ | BbhMovieRecommendation54BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT2 072 |
| ✓ | BbhMovieRecommendation55BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 210 |
| ✓ | BbhMovieRecommendation56BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT1 136 |
| ✓ | BbhMovieRecommendation57BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT2 220 |
| ✓ | BbhMovieRecommendation58BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 126 |
| ✓ | BbhMovieRecommendation59BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 435 |
| ✓ | BbhMovieRecommendation60BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 087 |
| ✓ | BbhMovieRecommendation61BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 849 |
| × | BbhMovieRecommendation62BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(B)")
Context: BbhMovieRecommendation62BT: expected (B), got [(C)] |
Failed | IN1 006 OUT6 372 |
| ✓ | BbhMovieRecommendation63BTFind a movie similar to Pretty Woman, Aladdin, The Lion King, Batman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT2 319 |
| × | BbhMovieRecommendation64BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(D)")
Context: BbhMovieRecommendation64BT: expected (D), got [(C)] |
Failed | IN1 007 OUT3 658 |
Reasoning - BBH movie recommendation (set 3)
30 passed |
100.0% |
IN30 288 OUT59 333 | |
| ✓ | BbhMovieRecommendation65BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 026 |
| ✓ | BbhMovieRecommendation66BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT2 642 |
| ✓ | BbhMovieRecommendation67BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 515 |
| ✓ | BbhMovieRecommendation68BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 786 |
| ✓ | BbhMovieRecommendation69BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 302 |
| ✓ | BbhMovieRecommendation70BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 710 |
| ✓ | BbhMovieRecommendation71BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT2 488 |
| ✓ | BbhMovieRecommendation72BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT2 522 |
| ✓ | BbhMovieRecommendation73BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT2 146 |
| ✓ | BbhMovieRecommendation74BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT4 369 |
| ✓ | BbhMovieRecommendation75BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 276 |
| ✓ | BbhMovieRecommendation76BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT2 238 |
| ✓ | BbhMovieRecommendation77BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 596 |
| ✓ | BbhMovieRecommendation78BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT2 058 |
| ✓ | BbhMovieRecommendation79BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 909 |
| ✓ | BbhMovieRecommendation80BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 833 |
| ✓ | BbhMovieRecommendation81BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT895 |
| ✓ | BbhMovieRecommendation82BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT2 207 |
| ✓ | BbhMovieRecommendation83BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 839 |
| ✓ | BbhMovieRecommendation84BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 929 |
| ✓ | BbhMovieRecommendation85BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 628 |
| ✓ | BbhMovieRecommendation86BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 625 |
| ✓ | BbhMovieRecommendation87BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 084 |
| ✓ | BbhMovieRecommendation88BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT2 559 |
| ✓ | BbhMovieRecommendation89BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 463 |
| ✓ | BbhMovieRecommendation90BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 777 |
| ✓ | BbhMovieRecommendation91BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 950 |
| ✓ | BbhMovieRecommendation92BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT3 339 |
| ✓ | BbhMovieRecommendation93BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT2 073 |
| ✓ | BbhMovieRecommendation94BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 549 |
Reasoning - BBH movie recommendation (set 4)
29 passed3 failed |
90.6% |
IN32 232 OUT62 460 | |
| ✓ | BbhMovieRecommendation100BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT2 222 |
| ✓ | BbhMovieRecommendation101BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 253 |
| ✓ | BbhMovieRecommendation102BUFind a movie similar to The Mask, Batman, Pretty Woman, The Fugitive:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 670 |
| ✓ | BbhMovieRecommendation103BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 703 |
| ✓ | BbhMovieRecommendation104BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 873 |
| ✓ | BbhMovieRecommendation105BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT2 302 |
| ✓ | BbhMovieRecommendation106BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT2 039 |
| ✓ | BbhMovieRecommendation107BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 228 |
| ✓ | BbhMovieRecommendation108BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT2 092 |
| × | BbhMovieRecommendation109BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation109BU: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 009 OUT2 532 |
| ✓ | BbhMovieRecommendation110BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT2 268 |
| × | BbhMovieRecommendation111BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation111BU: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 009 OUT2 608 |
| ✓ | BbhMovieRecommendation112BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 313 |
| ✓ | BbhMovieRecommendation113BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 890 |
| ✓ | BbhMovieRecommendation114BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 535 |
| ✓ | BbhMovieRecommendation115BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT2 594 |
| × | BbhMovieRecommendation116BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation116BU: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 018 OUT2 080 |
| ✓ | BbhMovieRecommendation117BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 450 |
| ✓ | BbhMovieRecommendation118BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 649 |
| ✓ | BbhMovieRecommendation119BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT2 192 |
| ✓ | BbhMovieRecommendation120BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 775 |
| ✓ | BbhMovieRecommendation121BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 289 |
| ✓ | BbhMovieRecommendation122BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 802 |
| ✓ | BbhMovieRecommendation123BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 357 |
| ✓ | BbhMovieRecommendation124BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT1 880 |
| ✓ | BbhMovieRecommendation125BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 523 |
| ✓ | BbhMovieRecommendation126BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 524 |
| ✓ | BbhMovieRecommendation95BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 033 |
| ✓ | BbhMovieRecommendation96BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 992 |
| ✓ | BbhMovieRecommendation97BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 186 |
| ✓ | BbhMovieRecommendation98BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT2 317 |
| ✓ | BbhMovieRecommendation99BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT3 289 |
Reasoning - BBH movie recommendation (set 5)
29 passed3 failed |
90.6% |
IN32 260 OUT63 749 | |
| ✓ | BbhMovieRecommendation127BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 187 |
| ✓ | BbhMovieRecommendation128BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 130 |
| ✓ | BbhMovieRecommendation129BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 655 |
| ✓ | BbhMovieRecommendation130BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 650 |
| ✓ | BbhMovieRecommendation131BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT2 111 |
| × | BbhMovieRecommendation132BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation132BU: expected (D), got [a]
Values:
got = "a" |
Failed | IN1 019 OUT1 943 |
| × | BbhMovieRecommendation133BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation133BU: expected (C), got [b]
Values:
got = "b" |
Failed | IN1 001 OUT2 688 |
| ✓ | BbhMovieRecommendation134BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 312 |
| ✓ | BbhMovieRecommendation135BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT2 670 |
| ✓ | BbhMovieRecommendation136BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 953 |
| ✓ | BbhMovieRecommendation137BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 102 |
| ✓ | BbhMovieRecommendation138BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 225 |
| ✓ | BbhMovieRecommendation139BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT2 700 |
| ✓ | BbhMovieRecommendation140BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT2 178 |
| ✓ | BbhMovieRecommendation141BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT2 566 |
| ✓ | BbhMovieRecommendation142BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 375 |
| ✓ | BbhMovieRecommendation143BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 165 |
| ✓ | BbhMovieRecommendation144BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT4 028 |
| ✓ | BbhMovieRecommendation145BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT2 634 |
| ✓ | BbhMovieRecommendation146BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 987 |
| ✓ | BbhMovieRecommendation147BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT2 481 |
| ✓ | BbhMovieRecommendation148BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 776 |
| ✓ | BbhMovieRecommendation149BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT1 052 |
| ✓ | BbhMovieRecommendation150BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 644 |
| ✓ | BbhMovieRecommendation151BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 159 |
| ✓ | BbhMovieRecommendation152BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 309 |
| ✓ | BbhMovieRecommendation153BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT1 642 |
| ✓ | BbhMovieRecommendation154BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT2 330 |
| × | BbhMovieRecommendation155BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation155BU: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 007 OUT932 |
| ✓ | BbhMovieRecommendation156BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 363 |
| ✓ | BbhMovieRecommendation157BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT2 797 |
| ✓ | BbhMovieRecommendation158BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT2 005 |
Reasoning - BBH movie recommendation (set 6)
29 passed3 failed |
90.6% |
IN32 200 OUT66 493 | |
| ✓ | BbhMovieRecommendation159BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 442 |
| ✓ | BbhMovieRecommendation160BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 045 |
| ✓ | BbhMovieRecommendation161BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 984 |
| ✓ | BbhMovieRecommendation162BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 545 |
| × | BbhMovieRecommendation163BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("Monsters, Inc")
Context: BbhMovieRecommendation163BU: expected Monsters, Inc, got [a]
Values:
got = "a" |
Failed | IN1 005 OUT6 436 |
| ✓ | BbhMovieRecommendation164BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 180 |
| ✓ | BbhMovieRecommendation165BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 319 |
| ✓ | BbhMovieRecommendation166BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT2 208 |
| ✓ | BbhMovieRecommendation167BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT2 445 |
| ✓ | BbhMovieRecommendation168BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 239 |
| ✓ | BbhMovieRecommendation169BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT2 617 |
| ✓ | BbhMovieRecommendation170BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 759 |
| ✓ | BbhMovieRecommendation171BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 057 |
| ✓ | BbhMovieRecommendation172BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 153 |
| ✓ | BbhMovieRecommendation173BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT2 680 |
| ✓ | BbhMovieRecommendation174BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 201 |
| ✓ | BbhMovieRecommendation175BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 352 |
| ✓ | BbhMovieRecommendation176BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 268 |
| × | BbhMovieRecommendation177BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation177BU: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 007 OUT3 956 |
| ✓ | BbhMovieRecommendation178BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 403 |
| ✓ | BbhMovieRecommendation179BUFind a movie similar to American Beauty, Pi, X-Men, The Godfather:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT2 712 |
| ✓ | BbhMovieRecommendation180BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT1 287 |
| ✓ | BbhMovieRecommendation181BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT1 321 |
| ✓ | BbhMovieRecommendation182BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 671 |
| ✓ | BbhMovieRecommendation183BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 298 |
| ✓ | BbhMovieRecommendation184BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT2 764 |
| ✓ | BbhMovieRecommendation185BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT2 051 |
| ✓ | BbhMovieRecommendation186BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT2 845 |
| ✓ | BbhMovieRecommendation187BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 232 |
| × | BbhMovieRecommendation188BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhMovieRecommendation188BU: expected (B), got [d]
Values:
got = "d" |
Failed | IN1 006 OUT3 847 |
| ✓ | BbhMovieRecommendation189BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 425 |
| ✓ | BbhMovieRecommendation190BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT751 |
Reasoning - BBH movie recommendation (set 7)
29 passed3 failed |
90.6% |
IN32 272 OUT62 381 | |
| ✓ | BbhMovieRecommendation191BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT2 385 |
| ✓ | BbhMovieRecommendation192BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 669 |
| ✓ | BbhMovieRecommendation193BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 040 |
| ✓ | BbhMovieRecommendation194BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT2 069 |
| × | BbhMovieRecommendation195BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhMovieRecommendation195BU: expected (A), got [d]
Values:
got = "d" |
Failed | IN996 OUT1 336 |
| ✓ | BbhMovieRecommendation196BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT2 366 |
| ✓ | BbhMovieRecommendation197BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT2 376 |
| ✓ | BbhMovieRecommendation198BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 848 |
| ✓ | BbhMovieRecommendation199BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 506 |
| ✓ | BbhMovieRecommendation200BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT1 998 |
| ✓ | BbhMovieRecommendation201BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 046 |
| ✓ | BbhMovieRecommendation202BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 430 |
| ✓ | BbhMovieRecommendation203BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 971 |
| ✓ | BbhMovieRecommendation204BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT2 199 |
| ✓ | BbhMovieRecommendation205BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 631 |
| ✓ | BbhMovieRecommendation206BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 661 |
| ✓ | BbhMovieRecommendation207BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT1 557 |
| ✓ | BbhMovieRecommendation208BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 804 |
| ✓ | BbhMovieRecommendation209BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 601 |
| ✓ | BbhMovieRecommendation210BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT2 671 |
| ✓ | BbhMovieRecommendation211BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 715 |
| ✓ | BbhMovieRecommendation212BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT2 455 |
| ✓ | BbhMovieRecommendation213BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 063 |
| ✓ | BbhMovieRecommendation214BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 623 |
| ✓ | BbhMovieRecommendation215BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 289 |
| ✓ | BbhMovieRecommendation216BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 996 |
| × | BbhMovieRecommendation217BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation217BU: expected (D), got [a]
Values:
got = "a" |
Failed | IN1 008 OUT3 711 |
| ✓ | BbhMovieRecommendation218BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 718 |
| ✓ | BbhMovieRecommendation219BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 752 |
| ✓ | BbhMovieRecommendation220BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 852 |
| ✓ | BbhMovieRecommendation221BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 414 |
| × | BbhMovieRecommendation222BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation222BU: expected (C), got [b]
Values:
got = "b" |
Failed | IN1 004 OUT3 629 |
Reasoning - BBH movie recommendation (set 8)
27 passed3 failed |
90.0% |
IN30 141 OUT54 063 | |
| ✓ | BbhMovieRecommendation223BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 299 |
| ✓ | BbhMovieRecommendation224BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT2 560 |
| ✓ | BbhMovieRecommendation225BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 079 |
| ✓ | BbhMovieRecommendation226BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT1 850 |
| ✓ | BbhMovieRecommendation227BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT2 239 |
| ✓ | BbhMovieRecommendation228BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 877 |
| ✓ | BbhMovieRecommendation229BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 794 |
| × | BbhMovieRecommendation230BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhMovieRecommendation230BU: expected (A), got [b]
Values:
got = "b" |
Failed | IN1 002 OUT2 084 |
| ✓ | BbhMovieRecommendation231BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT2 010 |
| ✓ | BbhMovieRecommendation232BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 370 |
| × | BbhMovieRecommendation233BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation233BU: expected (C), got [b]
Values:
got = "b" |
Failed | IN1 009 OUT2 173 |
| ✓ | BbhMovieRecommendation234BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 526 |
| ✓ | BbhMovieRecommendation235BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 978 |
| ✓ | BbhMovieRecommendation236BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 909 |
| ✓ | BbhMovieRecommendation237BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 704 |
| ✓ | BbhMovieRecommendation238BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 564 |
| ✓ | BbhMovieRecommendation239BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 510 |
| ✓ | BbhMovieRecommendation240BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 300 |
| ✓ | BbhMovieRecommendation241BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 627 |
| ✓ | BbhMovieRecommendation242BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT1 409 |
| ✓ | BbhMovieRecommendation243BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 680 |
| ✓ | BbhMovieRecommendation244BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 219 |
| ✓ | BbhMovieRecommendation245BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 133 |
| ✓ | BbhMovieRecommendation246BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT3 241 |
| × | BbhMovieRecommendation247BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation247BU: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 005 OUT2 027 |
| ✓ | BbhMovieRecommendation248BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT2 017 |
| ✓ | BbhMovieRecommendation249BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 021 |
| ✓ | BbhMultistepArithmeticTwo10BU((6 - 0 * 5 + -3) * (6 - -7 + -2 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 573 |
| ✓ | BbhMultistepArithmeticTwo8BU((-7 * -9 + 8 * -3) * (5 + -7 - 4 * -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 201 |
| ✓ | BbhMultistepArithmeticTwo9BU((-9 - 1 * 5 * -5) - (6 + -3 - -1 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 089 |
Reasoning - BBH multi-step arithmetic (set 1)
32 passed |
100.0% |
IN31 328 OUT41 316 | |
| ✓ | BbhMultistepArithmeticTwo11BV((2 - -2 + -7 * 8) * (-7 * -8 * 3 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 292 |
| ✓ | BbhMultistepArithmeticTwo12BV((8 - 2 + -2 * 6) * (8 + -6 + -8 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT910 |
| ✓ | BbhMultistepArithmeticTwo13BV((-6 + -9 - -6 + -4) * (-1 - -6 + -4 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 046 |
| ✓ | BbhMultistepArithmeticTwo14BV((-5 - 4 * -8 + 8) * (4 + 3 - 9 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 308 |
| ✓ | BbhMultistepArithmeticTwo15BV((-5 * -7 * -6 + 9) * (-2 - 8 + -5 + 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 419 |
| ✓ | BbhMultistepArithmeticTwo16BV((8 + 9 - 4 - -9) + (8 + 7 - 6 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 078 |
| ✓ | BbhMultistepArithmeticTwo17BV((5 * -1 + -6 * -3) + (-1 + -8 - 5 + 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT820 |
| ✓ | BbhMultistepArithmeticTwo18BV((-6 - 6 + 7 - 7) - (5 + 3 - 9 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 368 |
| ✓ | BbhMultistepArithmeticTwo19BV((7 - 4 + -3 * 4) - (5 + -8 - 6 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 072 |
| ✓ | BbhMultistepArithmeticTwo20BV((-6 * -1 - 2 + -2) + (9 - 4 + -1 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT885 |
| ✓ | BbhMultistepArithmeticTwo21BV((8 * -6 + 6 * 1) - (-3 * 7 * 0 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT950 |
| ✓ | BbhMultistepArithmeticTwo22BV((-6 - 8 - -7 * -2) - (-9 - 5 + 7 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 332 |
| ✓ | BbhMultistepArithmeticTwo23BV((1 - 7 - -8 * 3) + (-7 - -2 + -3 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 722 |
| ✓ | BbhMultistepArithmeticTwo24BV((5 * -8 - -5 * -9) * (2 - -7 * 6 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 957 |
| ✓ | BbhMultistepArithmeticTwo25BV((6 + 1 - 4 - 3) - (-4 * -6 * -3 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT981 |
| ✓ | BbhMultistepArithmeticTwo26BV((-9 - -9 + 0 + -3) + (-2 - -1 - 1 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 113 |
| ✓ | BbhMultistepArithmeticTwo27BV((9 - 3 + 2 + -1) - (5 - -1 - -6 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 603 |
| ✓ | BbhMultistepArithmeticTwo28BV((-8 + 6 * -2 + 4) * (-4 * 5 + 2 - 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT927 |
| ✓ | BbhMultistepArithmeticTwo29BV((3 + -1 * 7 * -6) - (-7 * -1 + -5 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 418 |
| ✓ | BbhMultistepArithmeticTwo30BV((3 - 3 - -3 + -2) + (8 - 2 + -8 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 420 |
| ✓ | BbhMultistepArithmeticTwo31BV((-2 + -9 - -2 * 2) - (6 * 1 * -2 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 991 |
| ✓ | BbhMultistepArithmeticTwo32BV((-7 + -2 * 1 - 6) + (3 + 9 * 7 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 014 |
| ✓ | BbhMultistepArithmeticTwo33BV((4 + 7 * 4 - -5) - (-4 - 1 - -4 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 066 |
| ✓ | BbhMultistepArithmeticTwo34BV((1 - 6 * 0 - -7) * (6 * 0 + 6 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT965 |
| ✓ | BbhMultistepArithmeticTwo35BV((-1 + -7 + -2 - 7) * (6 - 1 - -1 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 673 |
| ✓ | BbhMultistepArithmeticTwo36BV((-5 + 6 - -5 + -6) + (-4 + -6 - 3 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 050 |
| ✓ | BbhMultistepArithmeticTwo37BV((0 - -1 + -1 + -3) - (-3 * 0 * 8 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT777 |
| ✓ | BbhMultistepArithmeticTwo38BV((-8 + -6 * 9 - 7) + (8 + 2 * -9 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 558 |
| ✓ | BbhMultistepArithmeticTwo39BV((-3 * 0 * 4 - -7) + (-9 * -6 - -8 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 434 |
| ✓ | BbhMultistepArithmeticTwo40BV((-5 - 2 + 9 * -1) * (7 - 5 + 9 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 170 |
| ✓ | BbhMultistepArithmeticTwo41BV((5 - 7 - 8 - 6) * (-2 * -8 * -9 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 012 |
| ✓ | BbhMultistepArithmeticTwo42BV((-5 + 1 - 2 - -3) * (-9 * 3 + 9 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT985 |
Reasoning - BBH multi-step arithmetic (set 2)
31 passed1 failed |
96.9% |
IN31 328 OUT37 695 | |
| ✓ | BbhMultistepArithmeticTwo43BV((-2 * -8 * 9 + -5) + (7 - -1 - -4 - 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT718 |
| ✓ | BbhMultistepArithmeticTwo44BV((-2 - 5 * -5 - 6) + (-5 * -4 + 2 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 095 |
| ✓ | BbhMultistepArithmeticTwo45BV((-5 - 6 * 5 + -9) + (1 + -4 - -2 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 184 |
| ✓ | BbhMultistepArithmeticTwo46BV((-8 + 0 * 0 * -1) - (7 + -1 + -3 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT729 |
| ✓ | BbhMultistepArithmeticTwo47BV((-9 - -3 * 1 - -3) * (-9 - 5 - -2 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 361 |
| ✓ | BbhMultistepArithmeticTwo48BV((5 * -2 + 8 * 1) * (9 + -3 * -8 - -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 356 |
| ✓ | BbhMultistepArithmeticTwo49BV((-7 * 9 - -9 + 7) + (9 * -6 * -4 * -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT999 |
| ✓ | BbhMultistepArithmeticTwo50BV((4 - 3 * -1 * 3) * (3 + 8 + 1 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT819 |
| ✓ | BbhMultistepArithmeticTwo51BV((-3 - 8 * 5 + 5) - (-1 * -3 - -2 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 357 |
| ✓ | BbhMultistepArithmeticTwo52BV((1 + 3 - -5 - 4) + (-3 - -3 + 6 * 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT830 |
| ✓ | BbhMultistepArithmeticTwo53BV((3 * 9 - 3 * 2) + (-1 * -7 - 1 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT773 |
| ✓ | BbhMultistepArithmeticTwo54BV((7 * 9 * 3 * 6) * (-8 * -6 + -9 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 299 |
| ✓ | BbhMultistepArithmeticTwo55BV((1 + 1 + -7 * -6) - (4 * 2 * -5 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT949 |
| ✓ | BbhMultistepArithmeticTwo56BV((1 + 1 * 3 - 2) * (-8 - -2 - 9 + -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 416 |
| ✓ | BbhMultistepArithmeticTwo57BV((5 + 6 * 4 * 9) + (-9 - 0 * 3 - 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT931 |
| ✓ | BbhMultistepArithmeticTwo58BV((-1 - 9 + -9 * -9) - (1 + -3 - 5 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 230 |
| ✓ | BbhMultistepArithmeticTwo59BV((-1 + -6 - -4 + -6) - (-2 * -5 - -1 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT930 |
| ✓ | BbhMultistepArithmeticTwo60BV((-2 - -7 + 5 * -3) + (2 - 8 + 5 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT978 |
| ✓ | BbhMultistepArithmeticTwo61BV((3 - -6 - 7 + 3) + (-7 + 6 * 9 + 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 072 |
| ✓ | BbhMultistepArithmeticTwo62BV((-1 - -9 * 8 - -9) + (-9 - 2 * -6 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 503 |
| ✓ | BbhMultistepArithmeticTwo63BV((-5 + 6 + -6 * -1) + (-5 + -5 + 9 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 520 |
| ✓ | BbhMultistepArithmeticTwo64BV((4 + -1 + -7 + 2) - (-7 + -9 * -6 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT919 |
| ✓ | BbhMultistepArithmeticTwo65BV((8 * 9 * 0 - -1) - (-9 - -7 + -4 - 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 203 |
| ✓ | BbhMultistepArithmeticTwo66BV((4 + 5 * -9 * -8) - (-9 + -7 - -6 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 164 |
| ✓ | BbhMultistepArithmeticTwo67BV((-8 + 5 - 0 - 6) * (-5 * -4 - 6 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT854 |
| ✓ | BbhMultistepArithmeticTwo68BV((-4 + -5 * 7 + 7) + (3 * -3 - -4 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 036 |
| ✓ | BbhMultistepArithmeticTwo69BV((0 * 8 - -9 * 6) - (-3 * -9 * -1 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 330 |
| ✓ | BbhMultistepArithmeticTwo70BV((0 * 9 + -8 * 7) - (7 - 9 + 1 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT797 |
| × | BbhMultistepArithmeticTwo71BV((9 * 4 - 6 - -8) + (6 - 8 - -4 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "48")
Context: BbhMultistepArithmeticTwo71BV: expected 48, got [24] |
Failed | IN979 OUT3 063 |
| ✓ | BbhMultistepArithmeticTwo72BV((6 - 1 - -3 + -2) + (5 * 5 + -1 - -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT837 |
| ✓ | BbhMultistepArithmeticTwo73BV((-4 + 4 + 3 * 1) - (-9 - -3 + -6 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 122 |
| ✓ | BbhMultistepArithmeticTwo74BV((-3 - -6 - -7 + 3) + (5 * 5 * 1 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 321 |
Reasoning - BBH multi-step arithmetic (set 3)
32 passed |
100.0% |
IN31 328 OUT42 272 | |
| ✓ | BbhMultistepArithmeticTwo100BV((7 - 5 * 7 + -1) + (6 * 4 - 9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 421 |
| ✓ | BbhMultistepArithmeticTwo101BV((6 + -8 * -3 - -5) * (2 - 7 * -2 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 100 |
| ✓ | BbhMultistepArithmeticTwo102BV((-1 * -1 * -6 - 7) - (-4 * 0 - -4 - 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 232 |
| ✓ | BbhMultistepArithmeticTwo103BV((-6 + 1 - -4 * -2) + (4 - -8 - -1 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 040 |
| ✓ | BbhMultistepArithmeticTwo104BV((7 + -9 * -5 + -3) * (-8 * 8 - -5 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 329 |
| ✓ | BbhMultistepArithmeticTwo105BV((3 * 4 * 0 * -3) - (6 + -4 + 2 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT977 |
| ✓ | BbhMultistepArithmeticTwo106BV((2 + -6 * -7 - 7) + (-2 - -5 + -4 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT994 |
| ✓ | BbhMultistepArithmeticTwo75BV((-8 * 4 + 7 - 5) + (9 + -6 + -2 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 360 |
| ✓ | BbhMultistepArithmeticTwo76BV((-8 - -6 * 7 * 0) * (0 + -5 * -8 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 604 |
| ✓ | BbhMultistepArithmeticTwo77BV((5 * 1 * 3 - 9) * (-7 + 4 - -7 * -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 915 |
| ✓ | BbhMultistepArithmeticTwo78BV((5 - -2 * -9 + 8) - (3 - 4 * -3 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 164 |
| ✓ | BbhMultistepArithmeticTwo79BV((7 + 1 + 9 * 1) + (7 - -3 + 3 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT927 |
| ✓ | BbhMultistepArithmeticTwo80BV((-6 + -2 * 3 * 0) + (3 + -4 - 7 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT826 |
| ✓ | BbhMultistepArithmeticTwo81BV((-8 + -4 * -9 * 0) + (2 * -9 + 3 * -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 323 |
| ✓ | BbhMultistepArithmeticTwo82BV((-8 + 6 - 8 * -9) - (9 - 9 * -2 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 222 |
| ✓ | BbhMultistepArithmeticTwo83BV((9 - -6 * 9 * 4) - (6 - 4 * -8 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 326 |
| ✓ | BbhMultistepArithmeticTwo84BV((8 + 3 * -8 - -8) - (2 + 2 * -9 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 234 |
| ✓ | BbhMultistepArithmeticTwo85BV((-9 + 8 - -4 * 2) - (1 - 9 + -6 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 480 |
| ✓ | BbhMultistepArithmeticTwo86BV((-1 * -6 - -1 - -8) - (8 + -7 * 8 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 806 |
| ✓ | BbhMultistepArithmeticTwo87BV((-3 * 6 - 0 * 6) - (-6 * -6 * 7 + -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT956 |
| ✓ | BbhMultistepArithmeticTwo88BV((-6 - -1 - -5 * -5) + (8 + -5 - 9 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 400 |
| ✓ | BbhMultistepArithmeticTwo89BV((-9 + -4 + 0 * -1) * (4 * -3 * -8 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 168 |
| ✓ | BbhMultistepArithmeticTwo90BV((-2 - 2 * 2 + 0) - (7 - 6 - -7 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 122 |
| ✓ | BbhMultistepArithmeticTwo91BV((-7 - -4 * 2 - 9) * (9 - 6 - 1 + -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT935 |
| ✓ | BbhMultistepArithmeticTwo92BV((-9 * 4 * -9 - -9) * (5 * 2 * 6 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT889 |
| ✓ | BbhMultistepArithmeticTwo93BV((-9 + -3 + 6 - 8) - (-7 * -2 * -6 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT922 |
| ✓ | BbhMultistepArithmeticTwo94BV((-7 + 5 * -6 * -6) * (7 * -5 + -6 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 723 |
| ✓ | BbhMultistepArithmeticTwo95BV((-8 * 0 + 2 * 2) * (-9 - 1 * -9 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 472 |
| ✓ | BbhMultistepArithmeticTwo96BV((9 + 2 + 3 - -3) + (-5 + 6 - -6 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 022 |
| ✓ | BbhMultistepArithmeticTwo97BV((5 - 6 - 7 + 6) - (-1 - 1 * -7 + 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 075 |
| ✓ | BbhMultistepArithmeticTwo98BV((-9 - -7 + 2 + 2) + (-7 + -9 - -1 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT996 |
| ✓ | BbhMultistepArithmeticTwo99BV((-6 * -4 + -1 + 1) + (-9 * -6 + 7 + 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 312 |
Reasoning - BBH multi-step arithmetic (set 4)
31 passed |
100.0% |
IN30 349 OUT41 239 | |
| ✓ | BbhMultistepArithmeticTwo107BV((-3 + -4 * 5 * -7) - (-1 * -4 + -9 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 060 |
| ✓ | BbhMultistepArithmeticTwo108BV((0 + -9 * 6 * 4) + (5 - -3 + 8 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 054 |
| ✓ | BbhMultistepArithmeticTwo109BV((-4 - -6 - -5 - 5) - (-2 + -4 - -8 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 230 |
| ✓ | BbhMultistepArithmeticTwo110BV((-3 + -4 * 3 - -8) + (8 - 1 - -4 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT766 |
| ✓ | BbhMultistepArithmeticTwo111BV((-4 - 5 + 2 + 7) - (-9 * 6 * -1 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 082 |
| ✓ | BbhMultistepArithmeticTwo112BV((8 + 1 + 6 + 6) - (7 - -7 - 9 - -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 148 |
| ✓ | BbhMultistepArithmeticTwo113BV((-9 * -5 - 5 + 7) - (-3 - 8 + 9 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 448 |
| ✓ | BbhMultistepArithmeticTwo114BV((0 + 1 - -6 * 9) * (4 + 7 + -9 - 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 295 |
| ✓ | BbhMultistepArithmeticTwo115BV((-1 + -6 + -3 * -2) * (-9 * -1 + 2 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 126 |
| ✓ | BbhMultistepArithmeticTwo116BV((6 * 5 + 0 - -3) * (4 * 5 * 3 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT929 |
| ✓ | BbhMultistepArithmeticTwo117BV((-2 * -1 + 1 * -3) - (7 * -7 + -5 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 074 |
| ✓ | BbhMultistepArithmeticTwo118BV((-1 + 5 - -1 + 3) + (0 - -3 - -5 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 131 |
| ✓ | BbhMultistepArithmeticTwo119BV((0 - 5 + 0 + 3) * (-7 + -8 * 4 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 163 |
| ✓ | BbhMultistepArithmeticTwo120BV((-2 - 3 - 0 - 2) * (1 - -7 - -3 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 308 |
| ✓ | BbhMultistepArithmeticTwo121BV((3 + 3 - -4 - 2) + (-2 - -7 * -2 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 216 |
| ✓ | BbhMultistepArithmeticTwo122BV((-9 - -8 + 3 * -1) * (-9 * -2 * 4 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 258 |
| ✓ | BbhMultistepArithmeticTwo123BV((-1 - 9 * 6 + 3) * (0 - 8 - -4 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 312 |
| ✓ | BbhMultistepArithmeticTwo124BV((-9 - 8 * 9 - 0) + (-4 + 4 + -2 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT921 |
| ✓ | BbhMultistepArithmeticTwo125BV((-6 - -5 - -9 - -9) * (-6 * 4 - 8 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT987 |
| ✓ | BbhMultistepArithmeticTwo126BV((1 + 2 * -1 - -8) - (-8 - 2 - -1 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 469 |
| ✓ | BbhMultistepArithmeticTwo127BV((-6 * -2 + -2 - -9) + (-9 * 3 * -1 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT806 |
| ✓ | BbhMultistepArithmeticTwo128BV((-1 * -3 - 2 + -7) + (4 * 8 * -6 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 295 |
| ✓ | BbhMultistepArithmeticTwo129BV((8 + 6 * -1 - -3) * (-5 * 0 - -7 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 495 |
| ✓ | BbhMultistepArithmeticTwo130BV((-5 - -1 - 7 * 8) * (-1 - -4 * -7 * 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 584 |
| ✓ | BbhMultistepArithmeticTwo131BV((4 - 8 - -9 * 7) - (5 + -8 * 9 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 187 |
| ✓ | BbhMultistepArithmeticTwo132BV((-7 - -8 + 5 - -4) + (6 - -6 * 4 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT3 805 |
| ✓ | BbhMultistepArithmeticTwo133BV((-3 * 4 * 3 + -5) + (-7 - 5 + 7 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 157 |
| ✓ | BbhMultistepArithmeticTwo134BV((-2 - 1 - -6 + -9) - (4 * 8 + -8 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 273 |
| ✓ | BbhMultistepArithmeticTwo135BV((-2 * 4 - 3 - 3) - (-9 * -3 - -8 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 499 |
| ✓ | BbhMultistepArithmeticTwo136BV((-4 + -6 + -6 - -9) + (-2 * -5 * 5 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT894 |
| ✓ | BbhMultistepArithmeticTwo137BV((-5 * 5 + 1 - 4) + (9 * 0 - -7 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 267 |
Reasoning - BBH multi-step arithmetic (set 5)
31 passed |
100.0% |
IN30 349 OUT37 516 | |
| ✓ | BbhMultistepArithmeticTwo138BV((1 - -8 + 0 - -3) + (-3 + -4 + 1 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT751 |
| ✓ | BbhMultistepArithmeticTwo139BV((8 - -5 + -9 + 7) + (-9 * -1 * -6 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 150 |
| ✓ | BbhMultistepArithmeticTwo140BV((0 * 3 - -9 + -1) - (-5 * -6 - 6 + 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT918 |
| ✓ | BbhMultistepArithmeticTwo141BV((-4 + -9 + -7 * 8) - (-8 + 0 * -5 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 080 |
| ✓ | BbhMultistepArithmeticTwo142BV((7 + -7 * 9 * 5) - (-2 + -6 - -8 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT3 942 |
| ✓ | BbhMultistepArithmeticTwo143BV((-4 - -8 + 2 + -8) * (-3 * 0 + 4 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT728 |
| ✓ | BbhMultistepArithmeticTwo144BV((3 - 9 - 3 - 4) + (1 * -2 * -3 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 115 |
| ✓ | BbhMultistepArithmeticTwo145BV((-6 * -4 + 1 + -1) * (-1 + 2 * 7 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 301 |
| ✓ | BbhMultistepArithmeticTwo146BV((-9 * -3 - 8 - 5) + (1 * -2 - -4 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 854 |
| ✓ | BbhMultistepArithmeticTwo147BV((3 + -6 * -2 + 7) * (-6 - -4 - -6 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 496 |
| ✓ | BbhMultistepArithmeticTwo148BV((-7 * 2 + -4 * -8) - (-5 * 0 * 1 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT632 |
| ✓ | BbhMultistepArithmeticTwo149BV((-4 * 4 + -1 * 5) - (-6 + 1 - -8 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 388 |
| ✓ | BbhMultistepArithmeticTwo150BV((7 - -9 + 0 - 0) + (-9 + 5 - -9 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 126 |
| ✓ | BbhMultistepArithmeticTwo151BV((4 + 3 + -9 + -2) * (7 * 2 + 8 - 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT746 |
| ✓ | BbhMultistepArithmeticTwo152BV((3 - 1 + -3 * -4) - (8 * 9 - -7 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT969 |
| ✓ | BbhMultistepArithmeticTwo153BV((-6 * -7 * -8 - -2) + (-5 - 2 + 0 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT802 |
| ✓ | BbhMultistepArithmeticTwo154BV((5 * 7 * -4 + 7) * (-5 - -6 + -8 * -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 227 |
| ✓ | BbhMultistepArithmeticTwo155BV((4 - 2 * 2 + 3) + (-3 - -9 - -4 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 343 |
| ✓ | BbhMultistepArithmeticTwo156BV((8 - 7 - -3 + -9) - (-6 * -6 + -4 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 060 |
| ✓ | BbhMultistepArithmeticTwo157BV((-7 + 9 + -9 * -4) - (-7 * 6 - 7 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT923 |
| ✓ | BbhMultistepArithmeticTwo158BV((-6 * -2 + 6 - -8) - (-2 * -4 * 8 + 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 304 |
| ✓ | BbhMultistepArithmeticTwo159BV((-1 + 5 * 4 + 9) - (-7 - 6 * -8 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 885 |
| ✓ | BbhMultistepArithmeticTwo160BV((-4 + -2 + -6 + -1) + (-4 * 2 + -5 * 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT860 |
| ✓ | BbhMultistepArithmeticTwo161BV((-8 - -1 - -2 * -7) + (3 + 2 - 3 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 364 |
| ✓ | BbhMultistepArithmeticTwo162BV((4 + 5 - 7 + 9) * (-1 - -4 + -7 - -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT953 |
| ✓ | BbhMultistepArithmeticTwo163BV((7 + 7 * 3 - 3) - (-3 + 4 - -6 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 151 |
| ✓ | BbhMultistepArithmeticTwo164BV((-4 + 5 - 4 - -6) * (-3 * -1 * 2 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT878 |
| ✓ | BbhMultistepArithmeticTwo165BV((2 + 7 - -5 - 1) - (0 - 5 - 9 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT703 |
| ✓ | BbhMultistepArithmeticTwo166BV((4 * -4 + -6 - -2) - (9 + -7 + -8 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 652 |
| ✓ | BbhMultistepArithmeticTwo167BV((3 - 2 - 3 * 3) + (-8 * -1 * -9 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT999 |
| ✓ | BbhMultistepArithmeticTwo168BV((-1 * -5 - -3 + -9) + (-5 - 3 * -2 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 216 |
Reasoning - BBH multi-step arithmetic (set 6)
32 passed |
100.0% |
IN31 328 OUT38 784 | |
| ✓ | BbhMultistepArithmeticTwo169BW((5 * -1 - 7 * 5) - (6 - -5 * 3 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 827 |
| ✓ | BbhMultistepArithmeticTwo170BW((9 - -5 - 1 + -2) * (-5 * 3 - -5 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT925 |
| ✓ | BbhMultistepArithmeticTwo171BW((-8 - -3 * 5 + 4) * (2 * 7 - -3 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 284 |
| ✓ | BbhMultistepArithmeticTwo172BW((5 - 7 + -6 + 4) * (2 - -9 + 3 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 004 |
| ✓ | BbhMultistepArithmeticTwo173BW((-5 - -1 - 6 * 0) + (-3 + 5 * 0 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 287 |
| ✓ | BbhMultistepArithmeticTwo174BW((-3 - 9 - 3 - 1) + (-4 * -9 * -7 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 093 |
| ✓ | BbhMultistepArithmeticTwo175BW((1 * -1 + 5 * -4) + (-2 - 5 * 4 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 246 |
| ✓ | BbhMultistepArithmeticTwo176BW((-3 - -8 - -7 + -2) - (-2 - 8 + -1 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 251 |
| ✓ | BbhMultistepArithmeticTwo177BW((-5 * -6 + 3 * -3) - (-1 + -8 * 8 * -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 402 |
| ✓ | BbhMultistepArithmeticTwo178BW((-5 - -7 - 4 - 8) - (-4 + 4 + 1 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 010 |
| ✓ | BbhMultistepArithmeticTwo179BW((9 + 7 - 4 - 7) - (-5 + 6 + -1 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 028 |
| ✓ | BbhMultistepArithmeticTwo180BW((8 * -6 + -9 + 9) * (0 - 4 - 0 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 156 |
| ✓ | BbhMultistepArithmeticTwo181BW((-4 * 1 + 8 + 9) * (7 * -6 * -5 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 041 |
| ✓ | BbhMultistepArithmeticTwo182BW((-5 - 6 * 2 + 4) - (1 - 9 - 0 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 108 |
| ✓ | BbhMultistepArithmeticTwo183BW((-7 - 1 + 3 * -2) + (0 - 4 + 6 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 150 |
| ✓ | BbhMultistepArithmeticTwo184BW((7 * 2 + 4 - -6) * (6 - 7 - 7 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 635 |
| ✓ | BbhMultistepArithmeticTwo185BW((-6 * -9 + 1 * 7) - (0 + 3 * 5 - 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT656 |
| ✓ | BbhMultistepArithmeticTwo186BW((-8 - -1 + -8 + -3) * (-4 * 6 - 2 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 181 |
| ✓ | BbhMultistepArithmeticTwo187BW((0 * -4 - -7 * 5) + (-4 * 0 + 6 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 729 |
| ✓ | BbhMultistepArithmeticTwo188BW((9 * -9 + -4 * -8) - (5 * 0 + 8 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT724 |
| ✓ | BbhMultistepArithmeticTwo189BW((-6 + -9 - -6 + -1) + (-8 * 5 * -6 + 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT808 |
| ✓ | BbhMultistepArithmeticTwo190BW((5 + 4 + 7 - 6) - (-8 * 0 - 9 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT670 |
| ✓ | BbhMultistepArithmeticTwo191BW((9 - 6 + -4 + -5) * (4 - -8 + -9 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT774 |
| ✓ | BbhMultistepArithmeticTwo192BW((-9 * 6 - -3 * -3) + (-4 - -9 + 4 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 124 |
| ✓ | BbhMultistepArithmeticTwo193BW((-8 * -8 - -5 * -6) * (3 + -9 - 5 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 232 |
| ✓ | BbhMultistepArithmeticTwo194BW((0 + 3 * 4 * 9) - (4 - -5 + -4 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 125 |
| ✓ | BbhMultistepArithmeticTwo195BW((-4 + -7 - 3 + 3) - (-3 + -2 - -3 * -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 380 |
| ✓ | BbhMultistepArithmeticTwo196BW((-7 * -2 * -1 + -9) - (-7 + 5 - 0 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 012 |
| ✓ | BbhMultistepArithmeticTwo197BW((-3 + 2 * -7 * 5) * (-9 - 9 - -9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 422 |
| ✓ | BbhMultistepArithmeticTwo198BW((1 + 0 + 2 - 4) + (-9 + 6 * -5 + 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT850 |
| ✓ | BbhMultistepArithmeticTwo199BW((-3 + -4 * 4 + -8) + (1 - -3 + 4 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 163 |
| ✓ | BbhMultistepArithmeticTwo200BW((3 * -7 * -4 * 2) + (7 - -3 - -3 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 487 |
Reasoning - BBH multi-step arithmetic (set 7)
32 passed |
100.0% |
IN31 328 OUT36 072 | |
| ✓ | BbhMultistepArithmeticTwo201BW((-5 * -6 + -9 - 3) + (-7 - -2 - 6 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 706 |
| ✓ | BbhMultistepArithmeticTwo202BW((6 - 5 + 5 + -9) - (9 + 8 * -6 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT778 |
| ✓ | BbhMultistepArithmeticTwo203BW((8 * 3 - -1 - -3) * (-4 * 7 * -3 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 224 |
| ✓ | BbhMultistepArithmeticTwo204BW((5 + 3 - 6 + 1) + (-5 + 5 - 6 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT710 |
| ✓ | BbhMultistepArithmeticTwo205BW((-9 + 6 + 1 - 7) * (2 - -2 + -9 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT894 |
| ✓ | BbhMultistepArithmeticTwo206BW((1 * 0 - -5 - 3) + (9 * 6 + -4 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT882 |
| ✓ | BbhMultistepArithmeticTwo207BW((1 - 2 + 5 * 2) + (-2 - -7 + 3 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 151 |
| ✓ | BbhMultistepArithmeticTwo208BW((5 + -7 + 1 - 5) - (-3 * -1 * 7 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT891 |
| ✓ | BbhMultistepArithmeticTwo209BW((-7 * -2 + -4 + -4) - (9 * 1 * -6 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 255 |
| ✓ | BbhMultistepArithmeticTwo210BW((1 + 0 - 3 + 0) - (-7 - 9 * 1 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT852 |
| ✓ | BbhMultistepArithmeticTwo211BW((7 * 6 - -2 - 7) * (4 * 5 * -3 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 335 |
| ✓ | BbhMultistepArithmeticTwo212BW((3 * -9 * 3 - 7) - (-9 + 4 * 8 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 141 |
| ✓ | BbhMultistepArithmeticTwo213BW((9 + 6 - 9 + 8) * (9 - -9 * -6 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 403 |
| ✓ | BbhMultistepArithmeticTwo214BW((5 * 6 + -6 - 8) * (-5 - 6 - 5 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 288 |
| ✓ | BbhMultistepArithmeticTwo215BW((5 + 4 - -1 * -1) + (-6 - -7 + -1 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 279 |
| ✓ | BbhMultistepArithmeticTwo216BW((9 - 5 - 4 + -2) + (-9 + 1 - 1 - 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 094 |
| ✓ | BbhMultistepArithmeticTwo217BW((6 - 9 * -6 - -9) + (3 * 3 + -5 * -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 236 |
| ✓ | BbhMultistepArithmeticTwo218BW((-2 * 5 + -3 + 0) * (-9 + 2 + 5 * 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT859 |
| ✓ | BbhMultistepArithmeticTwo219BW((5 * -3 * 3 * 7) * (0 + -9 - -5 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT829 |
| ✓ | BbhMultistepArithmeticTwo220BW((8 - 3 + 0 + 7) + (-9 * -6 * 2 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT777 |
| ✓ | BbhMultistepArithmeticTwo221BW((-2 * 9 * -8 + 8) - (0 * -4 - -6 * 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 539 |
| ✓ | BbhMultistepArithmeticTwo222BW((-7 * -6 * 0 - 5) * (-3 - 9 * -8 * -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 490 |
| ✓ | BbhMultistepArithmeticTwo223BW((3 * 8 * -1 * -9) - (-2 + 2 * -8 * 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT927 |
| ✓ | BbhMultistepArithmeticTwo224BW((-9 + 5 * 0 - -8) + (6 - -5 * -7 + -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 574 |
| ✓ | BbhMultistepArithmeticTwo225BW((-2 - -7 - 5 * 2) + (-3 - 5 * 3 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 452 |
| ✓ | BbhMultistepArithmeticTwo226BW((-9 + -7 + -1 - -1) * (3 - -7 - -1 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 251 |
| ✓ | BbhMultistepArithmeticTwo227BW((-1 + 6 + 9 + 3) - (-5 - 9 - -3 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT604 |
| ✓ | BbhMultistepArithmeticTwo228BW((-7 + 9 * 9 * 6) - (-2 - -5 + 9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 216 |
| ✓ | BbhMultistepArithmeticTwo229BW((-8 - -7 + 0 + 3) + (-3 + 5 - -5 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT989 |
| ✓ | BbhMultistepArithmeticTwo230BW((8 - 4 - -8 - -7) + (6 - 5 - -1 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT754 |
| ✓ | BbhMultistepArithmeticTwo231BW((-6 * 8 + -6 - -5) + (-4 - 9 * 5 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT979 |
| ✓ | BbhMultistepArithmeticTwo232BW((5 * -9 - 4 - 4) - (-3 * 2 * 9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT713 |
Reasoning - BBH multi-step arithmetic (set 8)
32 passed |
100.0% |
IN31 687 OUT44 746 | |
| ✓ | BbhMultistepArithmeticTwo233BW((4 * -9 * 3 * -8) + (-9 - 4 - -9 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 020 |
| ✓ | BbhMultistepArithmeticTwo234BW((-7 - 2 - 0 + 2) * (-4 * -5 + -8 + 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT978 |
| ✓ | BbhMultistepArithmeticTwo235BW((-1 + 7 * 7 * -9) * (4 + 3 + -8 * 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 100 |
| ✓ | BbhMultistepArithmeticTwo236BW((7 * 8 + 1 + 3) - (3 + -1 - -9 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT978 |
| ✓ | BbhMultistepArithmeticTwo237BW((3 + -9 + 8 + -4) * (-4 + -6 - 4 - -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT700 |
| ✓ | BbhMultistepArithmeticTwo238BW((1 * 5 * -8 - -2) * (-2 * 6 - -5 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 790 |
| ✓ | BbhMultistepArithmeticTwo239BW((-8 - -1 * -8 * -2) * (-6 + 8 + -5 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 186 |
| ✓ | BbhMultistepArithmeticTwo240BW((4 - -6 + -9 + -7) - (9 + -3 - -7 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 730 |
| ✓ | BbhMultistepArithmeticTwo241BW((0 * -7 - 6 * 3) * (-9 + -5 - 2 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 150 |
| ✓ | BbhMultistepArithmeticTwo242BW((-8 * 1 * -1 + -5) + (-2 - -3 + 1 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT982 |
| ✓ | BbhMultistepArithmeticTwo243BW((8 - -9 * 4 * 9) * (-4 * -3 * 7 * 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT2 746 |
| ✓ | BbhMultistepArithmeticTwo244BW((1 * -9 - 4 + -9) + (-1 + -6 + 9 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT804 |
| ✓ | BbhMultistepArithmeticTwo245BW((-1 + 2 - 9 + 0) * (0 - -2 + -6 - 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 616 |
| ✓ | BbhMultistepArithmeticTwo246BW((7 * -5 * -9 + 7) - (2 + -8 * 6 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 002 |
| ✓ | BbhMultistepArithmeticTwo247BW((-5 + -8 - -6 * 7) + (9 * -7 - -5 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 666 |
| ✓ | BbhMultistepArithmeticTwo248BW((-6 * 9 - -5 + 8) * (-8 * -6 - -1 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 118 |
| ✓ | BbhMultistepArithmeticTwo249BW((5 - -2 * 7 + -2) - (0 * 9 * -6 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 116 |
| ✓ | BbhNavigate10BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 381 |
| ✓ | BbhNavigate11BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 675 |
| ✓ | BbhNavigate12BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 075 |
| ✓ | BbhNavigate13BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT896 |
| ✓ | BbhNavigate14BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT2 792 |
| ✓ | BbhNavigate15BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 129 |
| ✓ | BbhNavigate16BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT739 |
| ✓ | BbhNavigate17BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 429 |
| ✓ | BbhNavigate18BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT2 109 |
| ✓ | BbhNavigate19BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT2 362 |
| ✓ | BbhNavigate20BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 382 |
| ✓ | BbhNavigate21BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT961 |
| ✓ | BbhNavigate22BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 513 |
| ✓ | BbhNavigate8BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT1 652 |
| ✓ | BbhNavigate9BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT1 969 |
Reasoning - BBH navigation - returns to start? (set 1)
32 passed |
100.0% |
IN32 148 OUT45 273 | |
| ✓ | BbhNavigate23BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 344 |
| ✓ | BbhNavigate24BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 681 |
| ✓ | BbhNavigate25BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT2 468 |
| ✓ | BbhNavigate26BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT825 |
| ✓ | BbhNavigate27BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 292 |
| ✓ | BbhNavigate28BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 312 |
| ✓ | BbhNavigate29BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT816 |
| ✓ | BbhNavigate30BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 072 |
| ✓ | BbhNavigate31BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 279 |
| ✓ | BbhNavigate32BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT794 |
| ✓ | BbhNavigate33BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT2 038 |
| ✓ | BbhNavigate34BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 475 |
| ✓ | BbhNavigate35BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT1 192 |
| ✓ | BbhNavigate36BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT2 677 |
| ✓ | BbhNavigate37BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT965 |
| ✓ | BbhNavigate38BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT985 |
| ✓ | BbhNavigate39BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT663 |
| ✓ | BbhNavigate40BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 680 |
| ✓ | BbhNavigate41BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 160 |
| ✓ | BbhNavigate42BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 857 |
| ✓ | BbhNavigate43BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 346 |
| ✓ | BbhNavigate44BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 681 |
| ✓ | BbhNavigate45BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 285 |
| ✓ | BbhNavigate46BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 281 |
| ✓ | BbhNavigate47BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 139 |
| ✓ | BbhNavigate48BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT645 |
| ✓ | BbhNavigate49BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 788 |
| ✓ | BbhNavigate50BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT653 |
| ✓ | BbhNavigate51BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT714 |
| ✓ | BbhNavigate52BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT2 123 |
| ✓ | BbhNavigate53BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 681 |
| ✓ | BbhNavigate54BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT2 362 |
Reasoning - BBH navigation - returns to start? (set 2)
30 passed |
100.0% |
IN30 127 OUT43 598 | |
| ✓ | BbhNavigate55BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT783 |
| ✓ | BbhNavigate56BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 857 |
| ✓ | BbhNavigate57BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT849 |
| ✓ | BbhNavigate58BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 333 |
| ✓ | BbhNavigate59BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT2 656 |
| ✓ | BbhNavigate60BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT843 |
| ✓ | BbhNavigate61BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT4 437 |
| ✓ | BbhNavigate62BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 462 |
| ✓ | BbhNavigate63BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT2 646 |
| ✓ | BbhNavigate64BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT867 |
| ✓ | BbhNavigate65BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT637 |
| ✓ | BbhNavigate66BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 437 |
| ✓ | BbhNavigate67BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 256 |
| ✓ | BbhNavigate68BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 620 |
| ✓ | BbhNavigate69BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT736 |
| ✓ | BbhNavigate70BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT677 |
| ✓ | BbhNavigate71BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT644 |
| ✓ | BbhNavigate72BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 317 |
| ✓ | BbhNavigate73BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT637 |
| ✓ | BbhNavigate74BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT2 174 |
| ✓ | BbhNavigate75BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT692 |
| ✓ | BbhNavigate76BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 756 |
| ✓ | BbhNavigate77BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT1 733 |
| ✓ | BbhNavigate78BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT2 160 |
| ✓ | BbhNavigate79BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 075 |
| ✓ | BbhNavigate80BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 076 |
| ✓ | BbhNavigate81BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 523 |
| ✓ | BbhNavigate82BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT876 |
| ✓ | BbhNavigate83BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 587 |
| ✓ | BbhNavigate84BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 252 |
Reasoning - BBH navigation - returns to start? (set 3)
31 passed1 failed |
96.9% |
IN32 149 OUT57 092 | |
| ✓ | BbhNavigate100CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT925 |
| ✓ | BbhNavigate101CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 594 |
| ✓ | BbhNavigate102CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT939 |
| ✓ | BbhNavigate103CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT2 657 |
| ✓ | BbhNavigate104CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 783 |
| ✓ | BbhNavigate105CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 033 |
| ✓ | BbhNavigate106CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 187 |
| ✓ | BbhNavigate107CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT902 |
| ✓ | BbhNavigate108CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 764 |
| ✓ | BbhNavigate109CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 969 |
| ✓ | BbhNavigate110CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 681 |
| ✓ | BbhNavigate111CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT3 587 |
| ✓ | BbhNavigate112CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 981 |
| ✓ | BbhNavigate113CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT991 |
| ✓ | BbhNavigate114CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 404 |
| × | BbhNavigate115CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("Yes")
Context: BbhNavigate115CA: expected Yes, got [(A)] |
Failed | IN998 OUT3 057 |
| ✓ | BbhNavigate116CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 095 |
| ✓ | BbhNavigate85CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT2 877 |
| ✓ | BbhNavigate86CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 007 |
| ✓ | BbhNavigate87CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT808 |
| ✓ | BbhNavigate88CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 230 |
| ✓ | BbhNavigate89CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT3 177 |
| ✓ | BbhNavigate90CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT2 766 |
| ✓ | BbhNavigate91CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT1 192 |
| ✓ | BbhNavigate92CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 848 |
| ✓ | BbhNavigate93CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 567 |
| ✓ | BbhNavigate94CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 852 |
| ✓ | BbhNavigate95CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 128 |
| ✓ | BbhNavigate96CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 178 |
| ✓ | BbhNavigate97CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT3 148 |
| ✓ | BbhNavigate98CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT1 771 |
| ✓ | BbhNavigate99CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT994 |
Reasoning - BBH navigation - returns to start? (set 4)
31 passed1 failed |
96.9% |
IN32 137 OUT53 712 | |
| ✓ | BbhNavigate117CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 141 |
| ✓ | BbhNavigate118CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 083 |
| ✓ | BbhNavigate119CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT643 |
| ✓ | BbhNavigate120CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 245 |
| ✓ | BbhNavigate121CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT817 |
| ✓ | BbhNavigate122CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 431 |
| × | BbhNavigate123CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("No")
Context: BbhNavigate123CA: expected No, got [(B)] |
Failed | IN991 OUT2 254 |
| ✓ | BbhNavigate124CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 670 |
| ✓ | BbhNavigate125CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 481 |
| ✓ | BbhNavigate126CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT1 865 |
| ✓ | BbhNavigate127CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT2 069 |
| ✓ | BbhNavigate128CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 081 |
| ✓ | BbhNavigate129CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT752 |
| ✓ | BbhNavigate130CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT5 383 |
| ✓ | BbhNavigate131CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT2 267 |
| ✓ | BbhNavigate132CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 212 |
| ✓ | BbhNavigate133CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 164 |
| ✓ | BbhNavigate134CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT608 |
| ✓ | BbhNavigate135CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 833 |
| ✓ | BbhNavigate136CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 544 |
| ✓ | BbhNavigate137CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT3 884 |
| ✓ | BbhNavigate138CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 861 |
| ✓ | BbhNavigate139CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT3 416 |
| ✓ | BbhNavigate140CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 598 |
| ✓ | BbhNavigate141CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 779 |
| ✓ | BbhNavigate142CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 716 |
| ✓ | BbhNavigate143CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 240 |
| ✓ | BbhNavigate144CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT757 |
| ✓ | BbhNavigate145CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 424 |
| ✓ | BbhNavigate146CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT780 |
| ✓ | BbhNavigate147CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 617 |
| ✓ | BbhNavigate148CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 097 |
Reasoning - BBH navigation - returns to start? (set 5)
32 passed |
100.0% |
IN32 074 OUT49 773 | |
| ✓ | BbhNavigate149CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT2 184 |
| ✓ | BbhNavigate150CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 940 |
| ✓ | BbhNavigate151CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT835 |
| ✓ | BbhNavigate152CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT689 |
| ✓ | BbhNavigate153CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 697 |
| ✓ | BbhNavigate154CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 615 |
| ✓ | BbhNavigate155CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT568 |
| ✓ | BbhNavigate156CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 433 |
| ✓ | BbhNavigate157CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 950 |
| ✓ | BbhNavigate158CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 843 |
| ✓ | BbhNavigate159CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT718 |
| ✓ | BbhNavigate160CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT861 |
| ✓ | BbhNavigate161CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 158 |
| ✓ | BbhNavigate162CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT2 618 |
| ✓ | BbhNavigate163CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT848 |
| ✓ | BbhNavigate164CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 313 |
| ✓ | BbhNavigate165CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT959 |
| ✓ | BbhNavigate166CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT2 259 |
| ✓ | BbhNavigate167CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 128 |
| ✓ | BbhNavigate168CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT238 |
| ✓ | BbhNavigate169CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 118 |
| ✓ | BbhNavigate170CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT896 |
| ✓ | BbhNavigate171CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 018 |
| ✓ | BbhNavigate172CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT2 611 |
| ✓ | BbhNavigate173CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT3 564 |
| ✓ | BbhNavigate174CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT944 |
| ✓ | BbhNavigate175CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 750 |
| ✓ | BbhNavigate176CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 875 |
| ✓ | BbhNavigate177CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT1 468 |
| ✓ | BbhNavigate178CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 234 |
| ✓ | BbhNavigate179CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 551 |
| ✓ | BbhNavigate180CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT890 |
Reasoning - BBH navigation - returns to start? (set 6)
31 passed1 failed |
96.9% |
IN32 152 OUT44 855 | |
| ✓ | BbhNavigate181CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 219 |
| ✓ | BbhNavigate182CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 073 |
| ✓ | BbhNavigate183CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 713 |
| ✓ | BbhNavigate184CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT3 277 |
| ✓ | BbhNavigate185CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT765 |
| ✓ | BbhNavigate186CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT814 |
| ✓ | BbhNavigate187CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 597 |
| × | BbhNavigate188CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("Yes")
Context: BbhNavigate188CA: expected Yes, got [No] |
Failed | IN1 016 OUT1 644 |
| ✓ | BbhNavigate189CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT2 146 |
| ✓ | BbhNavigate190CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT697 |
| ✓ | BbhNavigate191CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 338 |
| ✓ | BbhNavigate192CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT1 474 |
| ✓ | BbhNavigate193CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 944 |
| ✓ | BbhNavigate194CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT872 |
| ✓ | BbhNavigate195CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT953 |
| ✓ | BbhNavigate196CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT1 632 |
| ✓ | BbhNavigate197CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 628 |
| ✓ | BbhNavigate198CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 443 |
| ✓ | BbhNavigate199CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 574 |
| ✓ | BbhNavigate200CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 444 |
| ✓ | BbhNavigate201CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 089 |
| ✓ | BbhNavigate202CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT762 |
| ✓ | BbhNavigate203CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 374 |
| ✓ | BbhNavigate204CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 986 |
| ✓ | BbhNavigate205CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 129 |
| ✓ | BbhNavigate206CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 638 |
| ✓ | BbhNavigate207CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 068 |
| ✓ | BbhNavigate208CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 216 |
| ✓ | BbhNavigate209CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 264 |
| ✓ | BbhNavigate210CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT529 |
| ✓ | BbhNavigate211CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 415 |
| ✓ | BbhNavigate212CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT2 138 |
Reasoning - BBH navigation - returns to start? (set 7)
30 passed |
100.0% |
IN30 053 OUT39 408 | |
| ✓ | BbhNavigate213CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 121 |
| ✓ | BbhNavigate214CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 491 |
| ✓ | BbhNavigate215CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 836 |
| ✓ | BbhNavigate216CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 307 |
| ✓ | BbhNavigate217CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 054 |
| ✓ | BbhNavigate218CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 595 |
| ✓ | BbhNavigate219CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 190 |
| ✓ | BbhNavigate220CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 005 |
| ✓ | BbhNavigate221CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT775 |
| ✓ | BbhNavigate222CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT766 |
| ✓ | BbhNavigate223CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT1 891 |
| ✓ | BbhNavigate224CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 083 |
| ✓ | BbhNavigate225CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT990 |
| ✓ | BbhNavigate226CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 574 |
| ✓ | BbhNavigate227CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT771 |
| ✓ | BbhNavigate228CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT839 |
| ✓ | BbhNavigate229CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT870 |
| ✓ | BbhNavigate230CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT865 |
| ✓ | BbhNavigate231CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 069 |
| ✓ | BbhNavigate232CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT769 |
| ✓ | BbhNavigate233CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 037 |
| ✓ | BbhNavigate234CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT3 005 |
| ✓ | BbhNavigate235CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT2 938 |
| ✓ | BbhNavigate236CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 218 |
| ✓ | BbhNavigate237CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT642 |
| ✓ | BbhNavigate238CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT874 |
| ✓ | BbhNavigate239CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 866 |
| ✓ | BbhNavigate240CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT964 |
| ✓ | BbhNavigate241CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT1 268 |
| ✓ | BbhNavigate242CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 735 |
Reasoning - BBH object counting (set 1)
32 passed |
100.0% |
IN31 706 OUT25 930 | |
| ✓ | BbhNavigate243CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 034 OUT1 488 |
| ✓ | BbhNavigate244CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT1 432 |
| ✓ | BbhNavigate245CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 173 |
| ✓ | BbhNavigate246CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT868 |
| ✓ | BbhNavigate247CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 349 |
| ✓ | BbhNavigate248CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT2 790 |
| ✓ | BbhNavigate249CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 326 |
| ✓ | BbhObjectCounting10CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT538 |
| ✓ | BbhObjectCounting11CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT820 |
| ✓ | BbhObjectCounting12CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT203 |
| ✓ | BbhObjectCounting13CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT546 |
| ✓ | BbhObjectCounting14CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT299 |
| ✓ | BbhObjectCounting15CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT355 |
| ✓ | BbhObjectCounting16CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT619 |
| ✓ | BbhObjectCounting17CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT407 |
| ✓ | BbhObjectCounting18CBI have a blackberry, a plum, and a banana. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT171 |
| ✓ | BbhObjectCounting19CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT275 |
| ✓ | BbhObjectCounting20CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT401 |
| ✓ | BbhObjectCounting21CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT966 |
| ✓ | BbhObjectCounting22CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT555 |
| ✓ | BbhObjectCounting23CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT921 |
| ✓ | BbhObjectCounting24CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT528 |
| ✓ | BbhObjectCounting25CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT510 |
| ✓ | BbhObjectCounting26CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT185 |
| ✓ | BbhObjectCounting27CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 132 |
| ✓ | BbhObjectCounting28CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT440 |
| ✓ | BbhObjectCounting29CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT399 |
| ✓ | BbhObjectCounting30CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT676 |
| ✓ | BbhObjectCounting31CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT548 |
| ✓ | BbhObjectCounting32CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT727 |
| ✓ | BbhObjectCounting8CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT720 |
| ✓ | BbhObjectCounting9CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT563 |
Reasoning - BBH object counting (set 2)
32 passed |
100.0% |
IN31 517 OUT17 024 | |
| ✓ | BbhObjectCounting33CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT270 |
| ✓ | BbhObjectCounting34CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN970 OUT241 |
| ✓ | BbhObjectCounting35CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT418 |
| ✓ | BbhObjectCounting36CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT310 |
| ✓ | BbhObjectCounting37CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT471 |
| ✓ | BbhObjectCounting38CBI have a raspberry, a grape, and an orange. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT323 |
| ✓ | BbhObjectCounting39CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT290 |
| ✓ | BbhObjectCounting40CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT472 |
| ✓ | BbhObjectCounting41CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT200 |
| ✓ | BbhObjectCounting42CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT398 |
| ✓ | BbhObjectCounting43CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT819 |
| ✓ | BbhObjectCounting44CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT338 |
| ✓ | BbhObjectCounting45CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT588 |
| ✓ | BbhObjectCounting46CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT581 |
| ✓ | BbhObjectCounting47CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT689 |
| ✓ | BbhObjectCounting48CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT454 |
| ✓ | BbhObjectCounting49CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT946 |
| ✓ | BbhObjectCounting50CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT655 |
| ✓ | BbhObjectCounting51CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT600 |
| ✓ | BbhObjectCounting52CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 168 |
| ✓ | BbhObjectCounting53CBI have a frog, and a mouse. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT224 |
| ✓ | BbhObjectCounting54CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT1 021 |
| ✓ | BbhObjectCounting55CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT704 |
| ✓ | BbhObjectCounting56CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT591 |
| ✓ | BbhObjectCounting57CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT514 |
| ✓ | BbhObjectCounting58CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT767 |
| ✓ | BbhObjectCounting59CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT467 |
| ✓ | BbhObjectCounting60CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT761 |
| ✓ | BbhObjectCounting61CBI have a rabbit, a frog, and a snail. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT164 |
| ✓ | BbhObjectCounting62CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT729 |
| ✓ | BbhObjectCounting63CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT491 |
| ✓ | BbhObjectCounting64CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT360 |
Reasoning - BBH object counting (set 3)
32 passed |
100.0% |
IN31 587 OUT17 149 | |
| ✓ | BbhObjectCounting65CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT403 |
| ✓ | BbhObjectCounting66CBI have a grape, and a plum. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT257 |
| ✓ | BbhObjectCounting67CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT635 |
| ✓ | BbhObjectCounting68CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT235 |
| ✓ | BbhObjectCounting69CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT216 |
| ✓ | BbhObjectCounting70CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT656 |
| ✓ | BbhObjectCounting71CBI have a dog, a pig, and a snail. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT297 |
| ✓ | BbhObjectCounting72CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT424 |
| ✓ | BbhObjectCounting73CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT610 |
| ✓ | BbhObjectCounting74CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 123 |
| ✓ | BbhObjectCounting75CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT786 |
| ✓ | BbhObjectCounting76CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 020 |
| ✓ | BbhObjectCounting77CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT409 |
| ✓ | BbhObjectCounting78CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT587 |
| ✓ | BbhObjectCounting79CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT395 |
| ✓ | BbhObjectCounting80CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT261 |
| ✓ | BbhObjectCounting81CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT382 |
| ✓ | BbhObjectCounting82CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT628 |
| ✓ | BbhObjectCounting83CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT701 |
| ✓ | BbhObjectCounting84CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT250 |
| ✓ | BbhObjectCounting85CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT671 |
| ✓ | BbhObjectCounting86CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT558 |
| ✓ | BbhObjectCounting87CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT743 |
| ✓ | BbhObjectCounting88CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT638 |
| ✓ | BbhObjectCounting89CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT592 |
| ✓ | BbhObjectCounting90CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT424 |
| ✓ | BbhObjectCounting91CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT588 |
| ✓ | BbhObjectCounting92CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT804 |
| ✓ | BbhObjectCounting93CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT438 |
| ✓ | BbhObjectCounting94CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT646 |
| ✓ | BbhObjectCounting95CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT346 |
| ✓ | BbhObjectCounting96CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT426 |
Reasoning - BBH object counting (set 4)
30 passed1 failed |
96.8% |
IN30 460 OUT17 783 | |
| ✓ | BbhObjectCounting100CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT1 134 |
| ✓ | BbhObjectCounting101CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT520 |
| ✓ | BbhObjectCounting102CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT726 |
| ✓ | BbhObjectCounting103CBI have a table, and three cars. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT345 |
| × | BbhObjectCounting104CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "8")
Context: BbhObjectCounting104CB: expected 8, got [7] |
Failed | IN981 OUT319 |
| ✓ | BbhObjectCounting105CBI have a microwave, a bed, and a couch. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT119 |
| ✓ | BbhObjectCounting106CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT248 |
| ✓ | BbhObjectCounting107CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 434 |
| ✓ | BbhObjectCounting108CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT642 |
| ✓ | BbhObjectCounting109CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT1 559 |
| ✓ | BbhObjectCounting110CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT386 |
| ✓ | BbhObjectCounting111CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT259 |
| ✓ | BbhObjectCounting112CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT985 |
| ✓ | BbhObjectCounting113CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT277 |
| ✓ | BbhObjectCounting114CBI have a donkey, a goat, and a cow. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT329 |
| ✓ | BbhObjectCounting115CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT251 |
| ✓ | BbhObjectCounting116CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT493 |
| ✓ | BbhObjectCounting117CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT478 |
| ✓ | BbhObjectCounting118CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT592 |
| ✓ | BbhObjectCounting119CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT626 |
| ✓ | BbhObjectCounting120CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT892 |
| ✓ | BbhObjectCounting121CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT179 |
| ✓ | BbhObjectCounting122CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT549 |
| ✓ | BbhObjectCounting123CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT774 |
| ✓ | BbhObjectCounting124CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT456 |
| ✓ | BbhObjectCounting125CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 135 |
| ✓ | BbhObjectCounting126CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT218 |
| ✓ | BbhObjectCounting127CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT400 |
| ✓ | BbhObjectCounting97CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT581 |
| ✓ | BbhObjectCounting98CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT285 |
| ✓ | BbhObjectCounting99CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT592 |
Reasoning - BBH object counting (set 5)
31 passed |
100.0% |
IN30 405 OUT13 269 | |
| ✓ | BbhObjectCounting128CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT484 |
| ✓ | BbhObjectCounting129CBI have two grapes, and a plum. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT421 |
| ✓ | BbhObjectCounting130CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT211 |
| ✓ | BbhObjectCounting131CBI have a pig, and a donkey. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN970 OUT305 |
| ✓ | BbhObjectCounting132CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT384 |
| ✓ | BbhObjectCounting133CBI have a microwave, and a bed. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT105 |
| ✓ | BbhObjectCounting134CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT257 |
| ✓ | BbhObjectCounting135CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT396 |
| ✓ | BbhObjectCounting136CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT229 |
| ✓ | BbhObjectCounting137CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT576 |
| ✓ | BbhObjectCounting138CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT230 |
| ✓ | BbhObjectCounting139CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT471 |
| ✓ | BbhObjectCounting140CBI have a chair, and two microwaves. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT237 |
| ✓ | BbhObjectCounting141CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT653 |
| ✓ | BbhObjectCounting142CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT556 |
| ✓ | BbhObjectCounting143CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT1 255 |
| ✓ | BbhObjectCounting144CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT619 |
| ✓ | BbhObjectCounting145CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT611 |
| ✓ | BbhObjectCounting146CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT481 |
| ✓ | BbhObjectCounting147CBI have a cat, and a fish. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT188 |
| ✓ | BbhObjectCounting148CBI have a head of broccoli, and a yam. How many vegetables do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT246 |
| ✓ | BbhObjectCounting149CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT446 |
| ✓ | BbhObjectCounting150CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT553 |
| ✓ | BbhObjectCounting151CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT119 |
| ✓ | BbhObjectCounting152CBI have four snails, and a cow. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN970 OUT481 |
| ✓ | BbhObjectCounting153CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT453 |
| ✓ | BbhObjectCounting154CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT141 |
| ✓ | BbhObjectCounting155CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT502 |
| ✓ | BbhObjectCounting156CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT589 |
| ✓ | BbhObjectCounting157CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT397 |
| ✓ | BbhObjectCounting158CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT673 |
Reasoning - BBH object counting (set 6)
31 passed1 failed |
96.9% |
IN31 480 OUT15 848 | |
| × | BbhObjectCounting159CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "8")
Context: BbhObjectCounting159CC: expected 8, got [6] |
Failed | IN987 OUT725 |
| ✓ | BbhObjectCounting160CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT423 |
| ✓ | BbhObjectCounting161CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT340 |
| ✓ | BbhObjectCounting162CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT448 |
| ✓ | BbhObjectCounting163CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT658 |
| ✓ | BbhObjectCounting164CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT325 |
| ✓ | BbhObjectCounting165CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 083 |
| ✓ | BbhObjectCounting166CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT405 |
| ✓ | BbhObjectCounting167CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT520 |
| ✓ | BbhObjectCounting168CCI have a toaster, a table, and a fridge. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT113 |
| ✓ | BbhObjectCounting169CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT302 |
| ✓ | BbhObjectCounting170CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT662 |
| ✓ | BbhObjectCounting171CCI have a garlic, and an onion. How many vegetables do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT220 |
| ✓ | BbhObjectCounting172CCI have a grape, a banana, and a blackberry. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT145 |
| ✓ | BbhObjectCounting173CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT622 |
| ✓ | BbhObjectCounting174CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT794 |
| ✓ | BbhObjectCounting175CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 348 |
| ✓ | BbhObjectCounting176CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT231 |
| ✓ | BbhObjectCounting177CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT336 |
| ✓ | BbhObjectCounting178CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT183 |
| ✓ | BbhObjectCounting179CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT644 |
| ✓ | BbhObjectCounting180CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT156 |
| ✓ | BbhObjectCounting181CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT358 |
| ✓ | BbhObjectCounting182CCI have a microwave, and a chair. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT145 |
| ✓ | BbhObjectCounting183CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT397 |
| ✓ | BbhObjectCounting184CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT365 |
| ✓ | BbhObjectCounting185CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT557 |
| ✓ | BbhObjectCounting186CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT868 |
| ✓ | BbhObjectCounting187CCI have a grape, an apple, and a banana. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT294 |
| ✓ | BbhObjectCounting188CCI have two blackberries, and a grape. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN970 OUT366 |
| ✓ | BbhObjectCounting189CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT703 |
| ✓ | BbhObjectCounting190CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT1 112 |
Reasoning - BBH object counting (set 7)
32 passed |
100.0% |
IN31 557 OUT19 098 | |
| ✓ | BbhObjectCounting191CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT900 |
| ✓ | BbhObjectCounting192CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT266 |
| ✓ | BbhObjectCounting193CCI have a dog, a chicken, and four snails. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT539 |
| ✓ | BbhObjectCounting194CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 054 |
| ✓ | BbhObjectCounting195CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT902 |
| ✓ | BbhObjectCounting196CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT205 |
| ✓ | BbhObjectCounting197CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT708 |
| ✓ | BbhObjectCounting198CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT840 |
| ✓ | BbhObjectCounting199CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 403 |
| ✓ | BbhObjectCounting200CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT706 |
| ✓ | BbhObjectCounting201CCI have an apple, and two blackberries. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN970 OUT307 |
| ✓ | BbhObjectCounting202CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT320 |
| ✓ | BbhObjectCounting203CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT710 |
| ✓ | BbhObjectCounting204CCI have two lamps, and two microwaves. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT284 |
| ✓ | BbhObjectCounting205CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT505 |
| ✓ | BbhObjectCounting206CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT917 |
| ✓ | BbhObjectCounting207CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT290 |
| ✓ | BbhObjectCounting208CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT631 |
| ✓ | BbhObjectCounting209CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 192 |
| ✓ | BbhObjectCounting210CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT174 |
| ✓ | BbhObjectCounting211CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT825 |
| ✓ | BbhObjectCounting212CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT482 |
| ✓ | BbhObjectCounting213CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT413 |
| ✓ | BbhObjectCounting214CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT587 |
| ✓ | BbhObjectCounting215CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT355 |
| ✓ | BbhObjectCounting216CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT861 |
| ✓ | BbhObjectCounting217CCI have a toaster, a car, and a table. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT212 |
| ✓ | BbhObjectCounting218CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 026 |
| ✓ | BbhObjectCounting219CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT608 |
| ✓ | BbhObjectCounting220CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT236 |
| ✓ | BbhObjectCounting221CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT216 |
| ✓ | BbhObjectCounting222CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT424 |
Reasoning - BBH object counting (set 8)
27 passed |
100.0% |
IN26 635 OUT15 340 | |
| ✓ | BbhObjectCounting223CCI have an oven, a bed, and a toaster. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT171 |
| ✓ | BbhObjectCounting224CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT858 |
| ✓ | BbhObjectCounting225CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT358 |
| ✓ | BbhObjectCounting226CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT272 |
| ✓ | BbhObjectCounting227CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT277 |
| ✓ | BbhObjectCounting228CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT757 |
| ✓ | BbhObjectCounting229CCI have two cars, and an oven. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN969 OUT290 |
| ✓ | BbhObjectCounting230CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT652 |
| ✓ | BbhObjectCounting231CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT861 |
| ✓ | BbhObjectCounting232CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT534 |
| ✓ | BbhObjectCounting233CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT655 |
| ✓ | BbhObjectCounting234CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT212 |
| ✓ | BbhObjectCounting235CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT659 |
| ✓ | BbhObjectCounting236CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT494 |
| ✓ | BbhObjectCounting237CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT949 |
| ✓ | BbhObjectCounting238CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT383 |
| ✓ | BbhObjectCounting239CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 684 |
| ✓ | BbhObjectCounting240CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT974 |
| ✓ | BbhObjectCounting241CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT182 |
| ✓ | BbhObjectCounting242CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT615 |
| ✓ | BbhObjectCounting243CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT307 |
| ✓ | BbhObjectCounting244CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT821 |
| ✓ | BbhObjectCounting245CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT513 |
| ✓ | BbhObjectCounting246CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT133 |
| ✓ | BbhObjectCounting247CCI have a fish, a snail, and two rabbits. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT431 |
| ✓ | BbhObjectCounting248CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT413 |
| ✓ | BbhObjectCounting249CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT885 |
Reasoning - BBH penguins in a table (set 1)
33 passed |
100.0% |
IN37 978 OUT27 672 | |
| ✓ | BbhPenguinsInATable10CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 018 |
| ✓ | BbhPenguinsInATable11CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 201 OUT569 |
| ✓ | BbhPenguinsInATable12CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT1 461 |
| ✓ | BbhPenguinsInATable13CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 224 OUT884 |
| ✓ | BbhPenguinsInATable14CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT451 |
| ✓ | BbhPenguinsInATable15CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT2 112 |
| ✓ | BbhPenguinsInATable16CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 226 OUT639 |
| ✓ | BbhPenguinsInATable17CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT502 |
| ✓ | BbhPenguinsInATable18CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT799 |
| ✓ | BbhPenguinsInATable19CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT303 |
| ✓ | BbhPenguinsInATable20CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT815 |
| ✓ | BbhPenguinsInATable21CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT843 |
| ✓ | BbhPenguinsInATable22CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT771 |
| ✓ | BbhPenguinsInATable23CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT983 |
| ✓ | BbhPenguinsInATable24CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT648 |
| ✓ | BbhPenguinsInATable25CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 121 OUT647 |
| ✓ | BbhPenguinsInATable26CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 218 OUT775 |
| ✓ | BbhPenguinsInATable27CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT1 378 |
| ✓ | BbhPenguinsInATable28CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT748 |
| ✓ | BbhPenguinsInATable29CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 220 OUT815 |
| ✓ | BbhPenguinsInATable30CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 218 OUT840 |
| ✓ | BbhPenguinsInATable31CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 225 OUT288 |
| ✓ | BbhPenguinsInATable32CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 193 OUT877 |
| ✓ | BbhPenguinsInATable33CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 195 OUT416 |
| ✓ | BbhPenguinsInATable34CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT3 072 |
| ✓ | BbhPenguinsInATable35CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT482 |
| ✓ | BbhPenguinsInATable36CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT644 |
| ✓ | BbhPenguinsInATable37CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT520 |
| ✓ | BbhPenguinsInATable38CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 199 OUT529 |
| ✓ | BbhPenguinsInATable39CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT468 |
| ✓ | BbhPenguinsInATable40CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT1 111 |
| ✓ | BbhPenguinsInATable8CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT469 |
| ✓ | BbhPenguinsInATable9CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT795 |
Reasoning - BBH penguins in a table (set 2)
34 passed |
100.0% |
IN38 629 OUT23 216 | |
| ✓ | BbhPenguinsInATable41CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT474 |
| ✓ | BbhPenguinsInATable42CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT728 |
| ✓ | BbhPenguinsInATable43CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 224 OUT1 618 |
| ✓ | BbhPenguinsInATable44CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT603 |
| ✓ | BbhPenguinsInATable45CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT576 |
| ✓ | BbhPenguinsInATable46CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 217 OUT374 |
| ✓ | BbhPenguinsInATable47CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 198 OUT377 |
| ✓ | BbhPenguinsInATable48CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT847 |
| ✓ | BbhPenguinsInATable49CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT628 |
| ✓ | BbhPenguinsInATable50CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 199 OUT725 |
| ✓ | BbhPenguinsInATable51CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT831 |
| ✓ | BbhPenguinsInATable52CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT509 |
| ✓ | BbhPenguinsInATable53CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT498 |
| ✓ | BbhPenguinsInATable54CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 138 OUT783 |
| ✓ | BbhPenguinsInATable55CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT457 |
| ✓ | BbhPenguinsInATable56CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT664 |
| ✓ | BbhPenguinsInATable57CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT745 |
| ✓ | BbhPenguinsInATable58CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT348 |
| ✓ | BbhPenguinsInATable59CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT755 |
| ✓ | BbhPenguinsInATable60CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT416 |
| ✓ | BbhPenguinsInATable61CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT518 |
| ✓ | BbhPenguinsInATable62CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT591 |
| ✓ | BbhPenguinsInATable63CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT524 |
| ✓ | BbhPenguinsInATable64CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 228 OUT1 243 |
| ✓ | BbhPenguinsInATable65CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 123 OUT477 |
| ✓ | BbhPenguinsInATable66CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT808 |
| ✓ | BbhPenguinsInATable67CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 223 OUT1 117 |
| ✓ | BbhPenguinsInATable68CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 153 |
| ✓ | BbhPenguinsInATable69CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT537 |
| ✓ | BbhPenguinsInATable70CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT576 |
| ✓ | BbhPenguinsInATable71CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT968 |
| ✓ | BbhPenguinsInATable72CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT577 |
| ✓ | BbhPenguinsInATable73CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT433 |
| ✓ | BbhPenguinsInATable74CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 222 OUT738 |
Reasoning - BBH penguins in a table (set 3)
36 passed |
100.0% |
IN41 003 OUT25 746 | |
| ✓ | BbhPenguins100CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT1 103 |
| ✓ | BbhPenguins101CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT566 |
| ✓ | BbhPenguins102CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT751 |
| ✓ | BbhPenguins103CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT415 |
| ✓ | BbhPenguins104CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 197 OUT915 |
| ✓ | BbhPenguins105CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT568 |
| ✓ | BbhPenguins106CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT515 |
| ✓ | BbhPenguins107CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT699 |
| ✓ | BbhPenguins108CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT799 |
| ✓ | BbhPenguins109CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT365 |
| ✓ | BbhPenguins110CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT606 |
| ✓ | BbhPenguins75CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 222 OUT691 |
| ✓ | BbhPenguins76CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT887 |
| ✓ | BbhPenguins77CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 197 OUT687 |
| ✓ | BbhPenguins78CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT673 |
| ✓ | BbhPenguins79CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT821 |
| ✓ | BbhPenguins80CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 193 OUT686 |
| ✓ | BbhPenguins81CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT796 |
| ✓ | BbhPenguins82CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT901 |
| ✓ | BbhPenguins83CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT627 |
| ✓ | BbhPenguins84CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 208 OUT825 |
| ✓ | BbhPenguins85CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT770 |
| ✓ | BbhPenguins86CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT387 |
| ✓ | BbhPenguins87CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT499 |
| ✓ | BbhPenguins88CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT1 256 |
| ✓ | BbhPenguins89CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 223 OUT1 408 |
| ✓ | BbhPenguins90CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT565 |
| ✓ | BbhPenguins91CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT548 |
| ✓ | BbhPenguins92CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 090 |
| ✓ | BbhPenguins93CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT514 |
| ✓ | BbhPenguins94CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT739 |
| ✓ | BbhPenguins95CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT574 |
| ✓ | BbhPenguins96CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 229 OUT901 |
| ✓ | BbhPenguins97CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT547 |
| ✓ | BbhPenguins98CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 190 OUT438 |
| ✓ | BbhPenguins99CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT614 |
Reasoning - BBH penguins in a table (set 4)
35 passed |
100.0% |
IN39 908 OUT24 915 | |
| ✓ | BbhPenguins111CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT497 |
| ✓ | BbhPenguins112CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT357 |
| ✓ | BbhPenguins113CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 198 OUT870 |
| ✓ | BbhPenguins114CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 200 OUT704 |
| ✓ | BbhPenguins115CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT504 |
| ✓ | BbhPenguins116CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT406 |
| ✓ | BbhPenguins117CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 225 OUT1 216 |
| ✓ | BbhPenguins118CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT773 |
| ✓ | BbhPenguins119CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT517 |
| ✓ | BbhPenguins120CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT967 |
| ✓ | BbhPenguins121CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 313 |
| ✓ | BbhPenguins122CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 204 OUT591 |
| ✓ | BbhPenguins123CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT957 |
| ✓ | BbhPenguins124CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT242 |
| ✓ | BbhPenguins125CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 192 OUT360 |
| ✓ | BbhPenguins126CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 422 |
| ✓ | BbhPenguins127CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT1 097 |
| ✓ | BbhPenguins128CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 198 OUT717 |
| ✓ | BbhPenguins129CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT637 |
| ✓ | BbhPenguins130CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT832 |
| ✓ | BbhPenguins131CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT561 |
| ✓ | BbhPenguins132CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 230 OUT777 |
| ✓ | BbhPenguins133CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 203 OUT696 |
| ✓ | BbhPenguins134CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT859 |
| ✓ | BbhPenguins135CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT418 |
| ✓ | BbhPenguins136CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT620 |
| ✓ | BbhPenguins137CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT442 |
| ✓ | BbhPenguins138CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT737 |
| ✓ | BbhPenguins139CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT725 |
| ✓ | BbhPenguins140CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT650 |
| ✓ | BbhPenguins141CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 223 OUT755 |
| ✓ | BbhPenguins142CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT622 |
| ✓ | BbhPenguins143CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT915 |
| ✓ | BbhPenguins144CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT529 |
| ✓ | BbhPenguins145CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT630 |
Reasoning - BBH reasoning about colored objects (set 1)
33 passed |
100.0% |
IN35 191 OUT26 634 | |
| ✓ | BbhColoredObjects10CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT663 |
| ✓ | BbhColoredObjects11CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT794 |
| ✓ | BbhColoredObjects12CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT670 |
| ✓ | BbhColoredObjects13CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT795 |
| ✓ | BbhColoredObjects14CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT936 |
| ✓ | BbhColoredObjects15CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT503 |
| ✓ | BbhColoredObjects16CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT796 |
| ✓ | BbhColoredObjects17CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT498 |
| ✓ | BbhColoredObjects18CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT804 |
| ✓ | BbhColoredObjects19CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT1 237 |
| ✓ | BbhColoredObjects20CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT660 |
| ✓ | BbhColoredObjects21CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT1 299 |
| ✓ | BbhColoredObjects22CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 645 |
| ✓ | BbhColoredObjects23CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT204 |
| ✓ | BbhColoredObjects24CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT1 633 |
| ✓ | BbhColoredObjects25CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT742 |
| ✓ | BbhColoredObjects26CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT569 |
| ✓ | BbhColoredObjects27CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT1 230 |
| ✓ | BbhColoredObjects28CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 886 |
| ✓ | BbhColoredObjects29CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT964 |
| ✓ | BbhColoredObjects30CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT466 |
| ✓ | BbhColoredObjects31CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT263 |
| ✓ | BbhColoredObjects32CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT622 |
| ✓ | BbhColoredObjects33CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT511 |
| ✓ | BbhColoredObjects34CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT371 |
| ✓ | BbhColoredObjects35CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT882 |
| ✓ | BbhColoredObjects36CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT460 |
| ✓ | BbhColoredObjects37CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT773 |
| ✓ | BbhColoredObjects38CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT890 |
| ✓ | BbhColoredObjects39CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT761 |
| ✓ | BbhColoredObjects40CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT865 |
| ✓ | BbhColoredObjects8CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT336 |
| ✓ | BbhColoredObjects9CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT906 |
Reasoning - BBH reasoning about colored objects (set 2)
30 passed |
100.0% |
IN31 782 OUT24 374 | |
| ✓ | BbhColoredObjects41CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT889 |
| ✓ | BbhColoredObjects42CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT686 |
| ✓ | BbhColoredObjects43CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT276 |
| ✓ | BbhColoredObjects44CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT531 |
| ✓ | BbhColoredObjects45CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT412 |
| ✓ | BbhColoredObjects46CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT492 |
| ✓ | BbhColoredObjects47CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT600 |
| ✓ | BbhColoredObjects48CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT668 |
| ✓ | BbhColoredObjects49CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 780 |
| ✓ | BbhColoredObjects50CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT798 |
| ✓ | BbhColoredObjects51CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT503 |
| ✓ | BbhColoredObjects52CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT959 |
| ✓ | BbhColoredObjects53CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT605 |
| ✓ | BbhColoredObjects54CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT457 |
| ✓ | BbhColoredObjects55CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT883 |
| ✓ | BbhColoredObjects56CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 350 |
| ✓ | BbhColoredObjects57CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT996 |
| ✓ | BbhColoredObjects58CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT844 |
| ✓ | BbhColoredObjects59CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT638 |
| ✓ | BbhColoredObjects60CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT1 092 |
| ✓ | BbhColoredObjects61CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 041 OUT2 182 |
| ✓ | BbhColoredObjects62CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT249 |
| ✓ | BbhColoredObjects63CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 232 |
| ✓ | BbhColoredObjects64CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT529 |
| ✓ | BbhColoredObjects65CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT1 140 |
| ✓ | BbhColoredObjects66CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT724 |
| ✓ | BbhColoredObjects67CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT642 |
| ✓ | BbhColoredObjects68CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT926 |
| ✓ | BbhColoredObjects69CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 048 |
| ✓ | BbhColoredObjects70CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT243 |
Reasoning - BBH reasoning about colored objects (set 3)
24 passed |
100.0% |
IN25 610 OUT19 743 | |
| ✓ | BbhColoredObjects71CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT358 |
| ✓ | BbhColoredObjects72CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT785 |
| ✓ | BbhColoredObjects73CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 218 |
| ✓ | BbhColoredObjects74CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT931 |
| ✓ | BbhColoredObjects75CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT1 767 |
| ✓ | BbhColoredObjects76CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT998 |
| ✓ | BbhColoredObjects77CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT261 |
| ✓ | BbhColoredObjects78CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 253 |
| ✓ | BbhColoredObjects79CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT846 |
| ✓ | BbhColoredObjects80CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT613 |
| ✓ | BbhColoredObjects81CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT753 |
| ✓ | BbhColoredObjects82CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT554 |
| ✓ | BbhColoredObjects83CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT194 |
| ✓ | BbhColoredObjects84CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT650 |
| ✓ | BbhColoredObjects85CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT1 175 |
| ✓ | BbhColoredObjects86CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT952 |
| ✓ | BbhColoredObjects87CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT781 |
| ✓ | BbhColoredObjects88CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT1 056 |
| ✓ | BbhColoredObjects89CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT514 |
| ✓ | BbhColoredObjects90CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT474 |
| ✓ | BbhColoredObjects91CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT878 |
| ✓ | BbhColoredObjects92CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 131 |
| ✓ | BbhColoredObjects93CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT688 |
| ✓ | BbhColoredObjects94CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT913 |
Reasoning - BBH reasoning about colored objects (set 4)
32 passed |
100.0% |
IN34 048 OUT24 343 | |
| ✓ | BbhReasoningAboutColoredObjects100CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT700 |
| ✓ | BbhReasoningAboutColoredObjects101CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT422 |
| ✓ | BbhReasoningAboutColoredObjects102CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT833 |
| ✓ | BbhReasoningAboutColoredObjects103CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT805 |
| ✓ | BbhReasoningAboutColoredObjects104CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT714 |
| ✓ | BbhReasoningAboutColoredObjects105CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 170 |
| ✓ | BbhReasoningAboutColoredObjects106CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT1 299 |
| ✓ | BbhReasoningAboutColoredObjects107CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 683 |
| ✓ | BbhReasoningAboutColoredObjects108CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT572 |
| ✓ | BbhReasoningAboutColoredObjects109CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT829 |
| ✓ | BbhReasoningAboutColoredObjects110CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT482 |
| ✓ | BbhReasoningAboutColoredObjects111CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT743 |
| ✓ | BbhReasoningAboutColoredObjects112CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT319 |
| ✓ | BbhReasoningAboutColoredObjects113CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT389 |
| ✓ | BbhReasoningAboutColoredObjects114CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT995 |
| ✓ | BbhReasoningAboutColoredObjects115CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT436 |
| ✓ | BbhReasoningAboutColoredObjects116CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT268 |
| ✓ | BbhReasoningAboutColoredObjects117CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT540 |
| ✓ | BbhReasoningAboutColoredObjects118CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT832 |
| ✓ | BbhReasoningAboutColoredObjects119CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT982 |
| ✓ | BbhReasoningAboutColoredObjects120CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT1 106 |
| ✓ | BbhReasoningAboutColoredObjects121CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT431 |
| ✓ | BbhReasoningAboutColoredObjects122CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT928 |
| ✓ | BbhReasoningAboutColoredObjects123CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT514 |
| ✓ | BbhReasoningAboutColoredObjects124CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT1 353 |
| ✓ | BbhReasoningAboutColoredObjects125CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT874 |
| ✓ | BbhReasoningAboutColoredObjects126CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT874 |
| ✓ | BbhReasoningAboutColoredObjects95CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT668 |
| ✓ | BbhReasoningAboutColoredObjects96CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT841 |
| ✓ | BbhReasoningAboutColoredObjects97CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT937 |
| ✓ | BbhReasoningAboutColoredObjects98CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT543 |
| ✓ | BbhReasoningAboutColoredObjects99CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT261 |
Reasoning - BBH reasoning about colored objects (set 5)
32 passed |
100.0% |
IN33 768 OUT26 479 | |
| ✓ | BbhReasoningAboutColoredObjects127CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT791 |
| ✓ | BbhReasoningAboutColoredObjects128CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT460 |
| ✓ | BbhReasoningAboutColoredObjects129CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT519 |
| ✓ | BbhReasoningAboutColoredObjects130CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT221 |
| ✓ | BbhReasoningAboutColoredObjects131CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT1 185 |
| ✓ | BbhReasoningAboutColoredObjects132CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 322 |
| ✓ | BbhReasoningAboutColoredObjects133CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT858 |
| ✓ | BbhReasoningAboutColoredObjects134CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT762 |
| ✓ | BbhReasoningAboutColoredObjects135CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT342 |
| ✓ | BbhReasoningAboutColoredObjects136CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT821 |
| ✓ | BbhReasoningAboutColoredObjects137CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT126 |
| ✓ | BbhReasoningAboutColoredObjects138CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT1 351 |
| ✓ | BbhReasoningAboutColoredObjects139CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT712 |
| ✓ | BbhReasoningAboutColoredObjects140CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT1 303 |
| ✓ | BbhReasoningAboutColoredObjects141CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT287 |
| ✓ | BbhReasoningAboutColoredObjects142CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT217 |
| ✓ | BbhReasoningAboutColoredObjects143CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT1 147 |
| ✓ | BbhReasoningAboutColoredObjects144CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 041 OUT869 |
| ✓ | BbhReasoningAboutColoredObjects145CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT1 047 |
| ✓ | BbhReasoningAboutColoredObjects146CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT725 |
| ✓ | BbhReasoningAboutColoredObjects147CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT1 049 |
| ✓ | BbhReasoningAboutColoredObjects148CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT1 148 |
| ✓ | BbhReasoningAboutColoredObjects149CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT751 |
| ✓ | BbhReasoningAboutColoredObjects150CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT1 417 |
| ✓ | BbhReasoningAboutColoredObjects151CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT962 |
| ✓ | BbhReasoningAboutColoredObjects152CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT236 |
| ✓ | BbhReasoningAboutColoredObjects153CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT597 |
| ✓ | BbhReasoningAboutColoredObjects154CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT739 |
| ✓ | BbhReasoningAboutColoredObjects155CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 201 |
| ✓ | BbhReasoningAboutColoredObjects156CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT1 754 |
| ✓ | BbhReasoningAboutColoredObjects157CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT1 291 |
| ✓ | BbhReasoningAboutColoredObjects158CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT269 |
Reasoning - BBH reasoning about colored objects (set 6)
32 passed |
100.0% |
IN34 011 OUT25 868 | |
| ✓ | BbhReasoningAboutColoredObjects159CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT459 |
| ✓ | BbhReasoningAboutColoredObjects160CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT1 044 |
| ✓ | BbhReasoningAboutColoredObjects161CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT635 |
| ✓ | BbhReasoningAboutColoredObjects162CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT362 |
| ✓ | BbhReasoningAboutColoredObjects163CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT244 |
| ✓ | BbhReasoningAboutColoredObjects164CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT410 |
| ✓ | BbhReasoningAboutColoredObjects165CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT330 |
| ✓ | BbhReasoningAboutColoredObjects166CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 584 |
| ✓ | BbhReasoningAboutColoredObjects167CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT944 |
| ✓ | BbhReasoningAboutColoredObjects168CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT799 |
| ✓ | BbhReasoningAboutColoredObjects169CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT756 |
| ✓ | BbhReasoningAboutColoredObjects170CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT385 |
| ✓ | BbhReasoningAboutColoredObjects171CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT1 236 |
| ✓ | BbhReasoningAboutColoredObjects172CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT625 |
| ✓ | BbhReasoningAboutColoredObjects173CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT631 |
| ✓ | BbhReasoningAboutColoredObjects174CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 034 OUT1 084 |
| ✓ | BbhReasoningAboutColoredObjects175CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT2 314 |
| ✓ | BbhReasoningAboutColoredObjects176CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT1 239 |
| ✓ | BbhReasoningAboutColoredObjects177CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT766 |
| ✓ | BbhReasoningAboutColoredObjects178CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT648 |
| ✓ | BbhReasoningAboutColoredObjects179CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 374 |
| ✓ | BbhReasoningAboutColoredObjects180CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT564 |
| ✓ | BbhReasoningAboutColoredObjects181CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT911 |
| ✓ | BbhReasoningAboutColoredObjects182CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 285 |
| ✓ | BbhReasoningAboutColoredObjects183CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT738 |
| ✓ | BbhReasoningAboutColoredObjects184CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT801 |
| ✓ | BbhReasoningAboutColoredObjects185CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT491 |
| ✓ | BbhReasoningAboutColoredObjects186CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT370 |
| ✓ | BbhReasoningAboutColoredObjects187CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT486 |
| ✓ | BbhReasoningAboutColoredObjects188CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT1 011 |
| ✓ | BbhReasoningAboutColoredObjects189CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT273 |
| ✓ | BbhReasoningAboutColoredObjects190CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT1 069 |
Reasoning - BBH reasoning about colored objects (set 7)
32 passed |
100.0% |
IN33 946 OUT21 341 | |
| ✓ | BbhReasoningAboutColoredObjects191CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT613 |
| ✓ | BbhReasoningAboutColoredObjects192CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT919 |
| ✓ | BbhReasoningAboutColoredObjects193CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT304 |
| ✓ | BbhReasoningAboutColoredObjects194CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT211 |
| ✓ | BbhReasoningAboutColoredObjects195CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT550 |
| ✓ | BbhReasoningAboutColoredObjects196CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT745 |
| ✓ | BbhReasoningAboutColoredObjects197CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT906 |
| ✓ | BbhReasoningAboutColoredObjects198CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT588 |
| ✓ | BbhReasoningAboutColoredObjects199CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT1 310 |
| ✓ | BbhReasoningAboutColoredObjects200CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 090 |
| ✓ | BbhReasoningAboutColoredObjects201CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT379 |
| ✓ | BbhReasoningAboutColoredObjects202CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT927 |
| ✓ | BbhReasoningAboutColoredObjects203CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT262 |
| ✓ | BbhReasoningAboutColoredObjects204CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT432 |
| ✓ | BbhReasoningAboutColoredObjects205CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT344 |
| ✓ | BbhReasoningAboutColoredObjects206CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT475 |
| ✓ | BbhReasoningAboutColoredObjects207CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT304 |
| ✓ | BbhReasoningAboutColoredObjects208CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT426 |
| ✓ | BbhReasoningAboutColoredObjects209CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT1 422 |
| ✓ | BbhReasoningAboutColoredObjects210CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT555 |
| ✓ | BbhReasoningAboutColoredObjects211CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT323 |
| ✓ | BbhReasoningAboutColoredObjects212CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT1 103 |
| ✓ | BbhReasoningAboutColoredObjects213CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT919 |
| ✓ | BbhReasoningAboutColoredObjects214CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT711 |
| ✓ | BbhReasoningAboutColoredObjects215CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT776 |
| ✓ | BbhReasoningAboutColoredObjects216CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT431 |
| ✓ | BbhReasoningAboutColoredObjects217CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT659 |
| ✓ | BbhReasoningAboutColoredObjects218CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT598 |
| ✓ | BbhReasoningAboutColoredObjects219CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT383 |
| ✓ | BbhReasoningAboutColoredObjects220CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT1 089 |
| ✓ | BbhReasoningAboutColoredObjects221CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT706 |
| ✓ | BbhReasoningAboutColoredObjects222CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT881 |
Reasoning - BBH reasoning about colored objects (set 8)
29 passed1 failed |
96.7% |
IN31 674 OUT28 359 | |
| ✓ | BbhReasoningAboutColoredObjects223CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 431 |
| ✓ | BbhReasoningAboutColoredObjects224CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT552 |
| ✓ | BbhReasoningAboutColoredObjects225CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT329 |
| ✓ | BbhReasoningAboutColoredObjects226CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT868 |
| ✓ | BbhReasoningAboutColoredObjects227CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT763 |
| ✓ | BbhReasoningAboutColoredObjects228CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT1 254 |
| ✓ | BbhReasoningAboutColoredObjects229CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT634 |
| ✓ | BbhReasoningAboutColoredObjects230CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT418 |
| ✓ | BbhReasoningAboutColoredObjects231CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT395 |
| ✓ | BbhReasoningAboutColoredObjects232CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT1 845 |
| ✓ | BbhReasoningAboutColoredObjects233CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT743 |
| ✓ | BbhReasoningAboutColoredObjects234CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT1 784 |
| ✓ | BbhReasoningAboutColoredObjects235CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT605 |
| ✓ | BbhReasoningAboutColoredObjects236CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT1 005 |
| ✓ | BbhReasoningAboutColoredObjects237CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT724 |
| ✓ | BbhReasoningAboutColoredObjects238CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT1 107 |
| ✓ | BbhReasoningAboutColoredObjects239CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT872 |
| ✓ | BbhReasoningAboutColoredObjects240CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT894 |
| ✓ | BbhReasoningAboutColoredObjects241CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT343 |
| ✓ | BbhReasoningAboutColoredObjects242CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT627 |
| ✓ | BbhReasoningAboutColoredObjects243CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 134 |
| ✓ | BbhReasoningAboutColoredObjects244CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT965 |
| ✓ | BbhReasoningAboutColoredObjects245CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT945 |
| ✓ | BbhReasoningAboutColoredObjects246CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT1 214 |
| ✓ | BbhReasoningAboutColoredObjects247CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT388 |
| ✓ | BbhReasoningAboutColoredObjects248CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT743 |
| ✓ | BbhReasoningAboutColoredObjects249CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT636 |
| × | BbhRuinNames10CEpick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames10CE: expected (A), got [d]
Values:
got = "d" |
Failed | IN996 OUT1 436 |
| ✓ | BbhRuinNames8CEpick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 103 |
| ✓ | BbhRuinNames9CEpick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 602 |
Reasoning - BBH ruin names (humorous edits) (set 1)
25 passed7 failed |
78.1% |
IN32 100 OUT43 816 | |
| ✓ | BbhRuinNames11CFWhich of the following is a humorous edit of this artist or movie name: 'the who!'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 311 |
| ✓ | BbhRuinNames12CFWhich of the following is a humorous edit of this artist or movie name: 'braveheart'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 354 |
| × | BbhRuinNames13CFWhich of the following is a humorous edit of this artist or movie name: 'the three stooges'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames13CF: expected (D), got [b]
Values:
got = "b" |
Failed | IN1 006 OUT1 703 |
| ✓ | BbhRuinNames14CFWhich of the following is a humorous edit of this artist or movie name: 'the deer hunter'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT710 |
| ✓ | BbhRuinNames15CFWhich of the following is a humorous edit of this artist or movie name: 'boyz 2 men'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 307 |
| ✓ | BbhRuinNames16CFWhich of the following is a humorous edit of this artist or movie name: 'the beastie boys'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT446 |
| ✓ | BbhRuinNames17CFWhich of the following is a humorous edit of this artist or movie name: 'joy division'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 527 |
| ✓ | BbhRuinNames18CFWhich of the following is a humorous edit of this artist or movie name: 'herbie hancock'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 639 |
| ✓ | BbhRuinNames19CFWhich of the following is a humorous edit of this artist or movie name: 'bon iver'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT3 590 |
| × | BbhRuinNames20CFWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames20CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN998 OUT451 |
| ✓ | BbhRuinNames21CFWhich of the following is a humorous edit of this artist or movie name: 'cinderella'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 026 |
| ✓ | BbhRuinNames22CFWhich of the following is a humorous edit of this artist or movie name: 'the italian job'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 573 |
| ✓ | BbhRuinNames23CFWhich of the following is a humorous edit of this artist or movie name: 'a few good men'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT772 |
| ✓ | BbhRuinNames24CFWhich of the following is a humorous edit of this artist or movie name: 'raiders of the lost ark'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 393 |
| ✓ | BbhRuinNames25CFWhich of the following is a humorous edit of this artist or movie name: 'little shop of horrors'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT909 |
| ✓ | BbhRuinNames26CFWhich of the following is a humorous edit of this artist or movie name: 'free willy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 822 |
| ✓ | BbhRuinNames27CFWhich of the following is a humorous edit of this artist or movie name: 'pixies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 376 |
| ✓ | BbhRuinNames28CFWhich of the following is a humorous edit of this artist or movie name: 'the princess and the frog'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 075 |
| ✓ | BbhRuinNames29CFWhich of the following is a humorous edit of this artist or movie name: 'foo fighters'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT767 |
| ✓ | BbhRuinNames30CFWhich of the following is a humorous edit of this artist or movie name: 'the godfather'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 056 |
| × | BbhRuinNames31CFWhich of the following is a humorous edit of this artist or movie name: 'pantera'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames31CF: expected (D), got [b]
Values:
got = "b" |
Failed | IN995 OUT1 776 |
| × | BbhRuinNames32CFWhich of the following is a humorous edit of this artist or movie name: 'top gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames32CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN995 OUT1 089 |
| ✓ | BbhRuinNames33CFWhich of the following is a humorous edit of this artist or movie name: 'die another day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT744 |
| ✓ | BbhRuinNames34CFWhich of the following is a humorous edit of this artist or movie name: 'the moody blues'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 520 |
| ✓ | BbhRuinNames35CFWhich of the following is a humorous edit of this artist or movie name: 'fall out boy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT910 |
| × | BbhRuinNames36CFWhich of the following is a humorous edit of this artist or movie name: 'love story'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames36CF: expected (A), got [c]
Values:
got = "c" |
Failed | IN998 OUT1 173 |
| ✓ | BbhRuinNames37CFWhich of the following is a humorous edit of this artist or movie name: 'whitney houston'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT3 020 |
| ✓ | BbhRuinNames38CFWhich of the following is a humorous edit of this artist or movie name: 'bridge of spies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 130 |
| × | BbhRuinNames39CFWhich of the following is a humorous edit of this artist or movie name: 'breaking benjamin'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames39CF: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 005 OUT1 033 |
| × | BbhRuinNames40CFWhich of the following is a humorous edit of this artist or movie name: 'frank kappa'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames40CF: expected (C), got [d]
Values:
got = "d" |
Failed | IN999 OUT2 262 |
| ✓ | BbhRuinNames41CFWhich of the following is a humorous edit of this artist or movie name: 'jurassic park'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT927 |
| ✓ | BbhRuinNames42CFWhich of the following is a humorous edit of this artist or movie name: 'the band'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 425 |
Reasoning - BBH ruin names (humorous edits) (set 2)
29 passed3 failed |
90.6% |
IN32 136 OUT47 048 | |
| ✓ | BbhRuinNames43CFWhich of the following is a humorous edit of this artist or movie name: 'creedence clearwater revival'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 147 |
| × | BbhRuinNames44CFWhich of the following is a humorous edit of this artist or movie name: 'block 57'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames44CF: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 008 OUT2 073 |
| ✓ | BbhRuinNames45CFWhich of the following is a humorous edit of this artist or movie name: 'get shorty'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 412 |
| ✓ | BbhRuinNames46CFWhich of the following is a humorous edit of this artist or movie name: 'radiohead'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 210 |
| ✓ | BbhRuinNames47CFWhich of the following is a humorous edit of this artist or movie name: 'mean girls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 295 |
| ✓ | BbhRuinNames48CFWhich of the following is a humorous edit of this artist or movie name: 'seven brides for seven brothers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT937 |
| ✓ | BbhRuinNames49CFWhich of the following is a humorous edit of this artist or movie name: 'the green mile'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 168 |
| ✓ | BbhRuinNames50CFWhich of the following is a humorous edit of this artist or movie name: 'cat on a hot tin roof'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT900 |
| ✓ | BbhRuinNames51CFWhich of the following is a humorous edit of this artist or movie name: 'new kids on the block'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 974 |
| ✓ | BbhRuinNames52CFWhich of the following is a humorous edit of this artist or movie name: 'superclunk'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 689 |
| ✓ | BbhRuinNames53CFWhich of the following is a humorous edit of this artist or movie name: 'my big fat greek wedding'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 042 |
| ✓ | BbhRuinNames54CFWhich of the following is a humorous edit of this artist or movie name: 'snakes on a plane'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT2 291 |
| × | BbhRuinNames55CFWhich of the following is a humorous edit of this artist or movie name: 'rocky'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames55CF: expected (A), got [d]
Values:
got = "d" |
Failed | IN995 OUT2 412 |
| ✓ | BbhRuinNames56CFWhich of the following is a humorous edit of this artist or movie name: 'pink floyd'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 209 |
| ✓ | BbhRuinNames57CFWhich of the following is a humorous edit of this artist or movie name: 'bad religion'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 437 |
| × | BbhRuinNames58CFWhich of the following is a humorous edit of this artist or movie name: 'the police'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames58CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN998 OUT1 880 |
| ✓ | BbhRuinNames59CFWhich of the following is a humorous edit of this artist or movie name: 'ice cube'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 419 |
| ✓ | BbhRuinNames60CFWhich of the following is a humorous edit of this artist or movie name: 'new order'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 624 |
| ✓ | BbhRuinNames61CFWhich of the following is a humorous edit of this artist or movie name: 'dolly dots'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 323 |
| ✓ | BbhRuinNames62CFWhich of the following is a humorous edit of this artist or movie name: 'die another day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 383 |
| ✓ | BbhRuinNames63CFWhich of the following is a humorous edit of this artist or movie name: 'mad max'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 042 |
| ✓ | BbhRuinNames64CFWhich of the following is a humorous edit of this artist or movie name: '8 mile'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 234 |
| ✓ | BbhRuinNames65CFWhich of the following is a humorous edit of this artist or movie name: 'i spit on your grave'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 072 |
| ✓ | BbhRuinNames66CFWhich of the following is a humorous edit of this artist or movie name: 'planet of the apes'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT896 |
| ✓ | BbhRuinNames67CFWhich of the following is a humorous edit of this artist or movie name: 'the rolling stones'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT772 |
| ✓ | BbhRuinNames68CFWhich of the following is a humorous edit of this artist or movie name: 'reservoir dogs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT971 |
| ✓ | BbhRuinNames69CFWhich of the following is a humorous edit of this artist or movie name: 'the men who stare at goats'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 491 |
| ✓ | BbhRuinNames70CFWhich of the following is a humorous edit of this artist or movie name: 'bridge of spies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 455 |
| ✓ | BbhRuinNames71CFWhich of the following is a humorous edit of this artist or movie name: 'last action hero'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 834 |
| ✓ | BbhRuinNames72CFWhich of the following is a humorous edit of this artist or movie name: 'the godfather'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 817 |
| ✓ | BbhRuinNames73CFWhich of the following is a humorous edit of this artist or movie name: 'the full monty'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 273 |
| ✓ | BbhRuinNames74CFWhich of the following is a humorous edit of this artist or movie name: 'fast and furious'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT3 366 |
Reasoning - BBH ruin names (humorous edits) (set 3)
27 passed5 failed |
84.4% |
IN32 159 OUT34 807 | |
| ✓ | BbhRuinNames100CFWhich of the following is a humorous edit of this artist or movie name: 'the battle of britain'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT512 |
| × | BbhRuinNames101CFWhich of the following is a humorous edit of this artist or movie name: 'dead man walking'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames101CF: expected (A), got [d]
Values:
got = "d" |
Failed | IN1 003 OUT1 232 |
| ✓ | BbhRuinNames102CFWhich of the following is a humorous edit of this artist or movie name: 'suicide squad'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT646 |
| × | BbhRuinNames103CFWhich of the following is a humorous edit of this artist or movie name: 'the dark knight'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames103CF: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 004 OUT1 316 |
| ✓ | BbhRuinNames104CFWhich of the following is a humorous edit of this artist or movie name: 'sleepless in seattle'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT810 |
| ✓ | BbhRuinNames105CFWhich of the following is a humorous edit of this artist or movie name: 'pink floyd'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 805 |
| ✓ | BbhRuinNames106CFWhich of the following is a humorous edit of this artist or movie name: 'the man with the golden gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT601 |
| ✓ | BbhRuinNames75CFWhich of the following is a humorous edit of this artist or movie name: 'skinny puppy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT765 |
| ✓ | BbhRuinNames76CFWhich of the following is a humorous edit of this artist or movie name: 'depeche mode'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT921 |
| ✓ | BbhRuinNames77CFWhich of the following is a humorous edit of this artist or movie name: 'sly and the family stone'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 487 |
| ✓ | BbhRuinNames78CFWhich of the following is a humorous edit of this artist or movie name: 'the departed'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT888 |
| ✓ | BbhRuinNames79CFWhich of the following is a humorous edit of this artist or movie name: 'blade runner'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 070 |
| ✓ | BbhRuinNames80CFWhich of the following is a humorous edit of this artist or movie name: 'the avengers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT858 |
| ✓ | BbhRuinNames81CFWhich of the following is a humorous edit of this artist or movie name: 'die hard'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 287 |
| ✓ | BbhRuinNames82CFWhich of the following is a humorous edit of this artist or movie name: 'weezer'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT755 |
| ✓ | BbhRuinNames83CFWhich of the following is a humorous edit of this artist or movie name: 'the longest day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 064 |
| × | BbhRuinNames84CFWhich of the following is a humorous edit of this artist or movie name: 'babs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames84CF: expected (A), got [d]
Values:
got = "d" |
Failed | IN994 OUT3 104 |
| ✓ | BbhRuinNames85CFWhich of the following is a humorous edit of this artist or movie name: 'moonlight'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 221 |
| ✓ | BbhRuinNames86CFWhich of the following is a humorous edit of this artist or movie name: 'animal farm'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT450 |
| ✓ | BbhRuinNames87CFWhich of the following is a humorous edit of this artist or movie name: 'seven brides for seven brothers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 013 |
| ✓ | BbhRuinNames88CFWhich of the following is a humorous edit of this artist or movie name: 'kiss'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 011 |
| ✓ | BbhRuinNames89CFWhich of the following is a humorous edit of this artist or movie name: 'the police'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT372 |
| ✓ | BbhRuinNames90CFWhich of the following is a humorous edit of this artist or movie name: 'en vogue'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 230 |
| ✓ | BbhRuinNames91CFWhich of the following is a humorous edit of this artist or movie name: 'top gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT736 |
| ✓ | BbhRuinNames92CFWhich of the following is a humorous edit of this artist or movie name: 'the girl with all the gifts'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT624 |
| ✓ | BbhRuinNames93CFWhich of the following is a humorous edit of this artist or movie name: 'the sisters of mercy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 203 |
| × | BbhRuinNames94CFWhich of the following is a humorous edit of this artist or movie name: 'the jam'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames94CF: expected (C), got [d]
Values:
got = "d" |
Failed | IN996 OUT990 |
| ✓ | BbhRuinNames95CFWhich of the following is a humorous edit of this artist or movie name: 'gone with the wind'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 703 |
| ✓ | BbhRuinNames96CFWhich of the following is a humorous edit of this artist or movie name: 'gone with the wind'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 018 |
| ✓ | BbhRuinNames97CFWhich of the following is a humorous edit of this artist or movie name: 'dream theater'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 868 |
| ✓ | BbhRuinNames98CFWhich of the following is a humorous edit of this artist or movie name: 'belle and sebastian'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT585 |
| × | BbhRuinNames99CFWhich of the following is a humorous edit of this artist or movie name: 'earth, wind, & fire'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("dearth, wind, & fire")
Context: BbhRuinNames99CF: expected dearth, wind, & fire, got [i]
Values:
got = "i" |
Failed | IN1 031 OUT1 662 |
Reasoning - BBH ruin names (humorous edits) (set 4)
28 passed4 failed |
87.5% |
IN32 143 OUT47 837 | |
| ✓ | BbhRuinNames107CFWhich of the following is a humorous edit of this artist or movie name: 'the mash'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 027 |
| ✓ | BbhRuinNames108CFWhich of the following is a humorous edit of this artist or movie name: 'missy elliott'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT784 |
| ✓ | BbhRuinNames109CFWhich of the following is a humorous edit of this artist or movie name: 'sleater-kinney'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 451 |
| ✓ | BbhRuinNames110CFWhich of the following is a humorous edit of this artist or movie name: 'dirty dancing'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 910 |
| × | BbhRuinNames111CFWhich of the following is a humorous edit of this artist or movie name: 'jay and silent bob strike back'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames111CF: expected (D), got [b]
Values:
got = "b" |
Failed | IN1 020 OUT1 521 |
| ✓ | BbhRuinNames112CFWhich of the following is a humorous edit of this artist or movie name: 'a bridge too far'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 573 |
| ✓ | BbhRuinNames113CFWhich of the following is a humorous edit of this artist or movie name: 'the beatles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 357 |
| × | BbhRuinNames114CFWhich of the following is a humorous edit of this artist or movie name: 'night club'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames114CF: expected (A), got [b]
Values:
got = "b" |
Failed | IN1 000 OUT2 746 |
| ✓ | BbhRuinNames115CFWhich of the following is a humorous edit of this artist or movie name: 'no time to die'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT769 |
| × | BbhRuinNames116CFWhich of the following is a humorous edit of this artist or movie name: 'batman begins'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames116CF: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 001 OUT693 |
| ✓ | BbhRuinNames117CFWhich of the following is a humorous edit of this artist or movie name: 'gold finger'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 661 |
| ✓ | BbhRuinNames118CFWhich of the following is a humorous edit of this artist or movie name: 'calamity jane'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 321 |
| ✓ | BbhRuinNames119CFWhich of the following is a humorous edit of this artist or movie name: 'saving private ryan'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT971 |
| ✓ | BbhRuinNames120CFWhich of the following is a humorous edit of this artist or movie name: 'the cult'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 687 |
| ✓ | BbhRuinNames121CFWhich of the following is a humorous edit of this artist or movie name: 'johnny got his gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT2 088 |
| ✓ | BbhRuinNames122CFWhich of the following is a humorous edit of this artist or movie name: 'counting crows'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 878 |
| ✓ | BbhRuinNames123CFWhich of the following is a humorous edit of this artist or movie name: 'lawrence of arabia'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 331 |
| ✓ | BbhRuinNames124CFWhich of the following is a humorous edit of this artist or movie name: 'jungle book'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT930 |
| ✓ | BbhRuinNames125CFWhich of the following is a humorous edit of this artist or movie name: 'paint your wagon'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT2 443 |
| ✓ | BbhRuinNames126CFWhich of the following is a humorous edit of this artist or movie name: 'blade'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT2 206 |
| ✓ | BbhRuinNames127CFWhich of the following is a humorous edit of this artist or movie name: 'the sisterhood of the traveling pants'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT1 035 |
| ✓ | BbhRuinNames128CFWhich of the following is a humorous edit of this artist or movie name: 'green day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 121 |
| ✓ | BbhRuinNames129CFWhich of the following is a humorous edit of this artist or movie name: 'the sting'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT768 |
| ✓ | BbhRuinNames130CFWhich of the following is a humorous edit of this artist or movie name: 'the artist'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 758 |
| ✓ | BbhRuinNames131CFWhich of the following is a humorous edit of this artist or movie name: 'the hope'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 326 |
| × | BbhRuinNames132CFWhich of the following is a humorous edit of this artist or movie name: 'blade runner'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames132CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN1 000 OUT3 497 |
| ✓ | BbhRuinNames133CFWhich of the following is a humorous edit of this artist or movie name: 'a view to a kill'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT518 |
| ✓ | BbhRuinNames134CFWhich of the following is a humorous edit of this artist or movie name: 'hope floats'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT935 |
| ✓ | BbhRuinNames135CFWhich of the following is a humorous edit of this artist or movie name: 'the apartment'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT2 109 |
| ✓ | BbhRuinNames136CFWhich of the following is a humorous edit of this artist or movie name: 'schindler's list'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT847 |
| ✓ | BbhRuinNames137CFWhich of the following is a humorous edit of this artist or movie name: 'beauty and the beast'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT856 |
| ✓ | BbhRuinNames138CFWhich of the following is a humorous edit of this artist or movie name: 'news of the world'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT720 |
Reasoning - BBH ruin names (humorous edits) (set 5)
25 passed5 failed |
83.3% |
IN30 139 OUT37 913 | |
| ✓ | BbhRuinNames139CFWhich of the following is a humorous edit of this artist or movie name: 'the guns of navarone'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT835 |
| ✓ | BbhRuinNames140CFWhich of the following is a humorous edit of this artist or movie name: 'the beatles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 120 |
| ✓ | BbhRuinNames141CFWhich of the following is a humorous edit of this artist or movie name: 'the godfather'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT473 |
| ✓ | BbhRuinNames142CFWhich of the following is a humorous edit of this artist or movie name: 'hank williams'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT974 |
| ✓ | BbhRuinNames143CFWhich of the following is a humorous edit of this artist or movie name: 'black sabbath'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 379 |
| × | BbhRuinNames144CFWhich of the following is a humorous edit of this artist or movie name: 'rita, sue and bob too'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("rita, sue and bob poo")
Context: BbhRuinNames144CF: expected rita, sue and bob poo, got [d]
Values:
got = "d" |
Failed | IN1 025 OUT1 153 |
| ✓ | BbhRuinNames145CFWhich of the following is a humorous edit of this artist or movie name: 'jethro tull'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT673 |
| ✓ | BbhRuinNames146CFWhich of the following is a humorous edit of this artist or movie name: 'the sound of music'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT654 |
| ✓ | BbhRuinNames147CFWhich of the following is a humorous edit of this artist or movie name: 'dream theater'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 260 |
| ✓ | BbhRuinNames148CFWhich of the following is a humorous edit of this artist or movie name: 'my fair lady'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 170 |
| ✓ | BbhRuinNames149CFWhich of the following is a humorous edit of this artist or movie name: 'offspring'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 932 |
| × | BbhRuinNames150CFWhich of the following is a humorous edit of this artist or movie name: 'the longest day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames150CF: expected (A), got [b]
Values:
got = "b" |
Failed | IN1 004 OUT3 097 |
| ✓ | BbhRuinNames151CFWhich of the following is a humorous edit of this artist or movie name: 'apocalypse now'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT731 |
| ✓ | BbhRuinNames152CFWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT3 209 |
| ✓ | BbhRuinNames153CFWhich of the following is a humorous edit of this artist or movie name: 'big country'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 138 |
| ✓ | BbhRuinNames154CFWhich of the following is a humorous edit of this artist or movie name: '12 angry men'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT513 |
| ✓ | BbhRuinNames155CFWhich of the following is a humorous edit of this artist or movie name: 'in search of a midnight kiss'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT868 |
| ✓ | BbhRuinNames156CFWhich of the following is a humorous edit of this artist or movie name: 'men in black'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 522 |
| ✓ | BbhRuinNames157CFWhich of the following is a humorous edit of this artist or movie name: 'the beatles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT801 |
| ✓ | BbhRuinNames158CFWhich of the following is a humorous edit of this artist or movie name: 'raging bull'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 080 |
| × | BbhRuinNames159CFWhich of the following is a humorous edit of this artist or movie name: 'bad brains'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames159CF: expected (D), got [c]
Values:
got = "c" |
Failed | IN999 OUT1 412 |
| ✓ | BbhRuinNames160CFWhich of the following is a humorous edit of this artist or movie name: 'thin lizzy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 202 |
| ✓ | BbhRuinNames161CFWhich of the following is a humorous edit of this artist or movie name: 'ben folds five'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 046 |
| ✓ | BbhRuinNames162CFWhich of the following is a humorous edit of this artist or movie name: 'citizen kane'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 210 |
| ✓ | BbhRuinNames163CFWhich of the following is a humorous edit of this artist or movie name: 'dave matthews band'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 404 |
| ✓ | BbhRuinNames164CFWhich of the following is a humorous edit of this artist or movie name: 'the sum of all fears'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 396 |
| ✓ | BbhRuinNames165CFWhich of the following is a humorous edit of this artist or movie name: 'city hall'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 359 |
| × | BbhRuinNames166CFWhich of the following is a humorous edit of this artist or movie name: 'the abyss'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames166CF: expected (A), got [c]
Values:
got = "c" |
Failed | IN999 OUT2 047 |
| × | BbhRuinNames167CFWhich of the following is a humorous edit of this artist or movie name: 'mank'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames167CF: expected (B), got [d]
Values:
got = "d" |
Failed | IN993 OUT1 362 |
| ✓ | BbhRuinNames168CFWhich of the following is a humorous edit of this artist or movie name: 'sex pistols'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT893 |
Reasoning - BBH ruin names (humorous edits) (set 6)
38 passed3 failed |
92.7% |
IN41 193 OUT49 950 | |
| ✓ | BbhRuinNames169CGWhich of the following is a humorous edit of this artist or movie name: 'life of brian'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 733 |
| ✓ | BbhRuinNames170CGWhich of the following is a humorous edit of this artist or movie name: 'black panther'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 083 |
| ✓ | BbhRuinNames171CGWhich of the following is a humorous edit of this artist or movie name: 'goldfinger'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 677 |
| ✓ | BbhRuinNames172CGWhich of the following is a humorous edit of this artist or movie name: 'foo fighters'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 250 |
| ✓ | BbhRuinNames173CGWhich of the following is a humorous edit of this artist or movie name: 'air supply'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT960 |
| ✓ | BbhRuinNames174CGWhich of the following is a humorous edit of this artist or movie name: 'diana ross and the supremes'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT1 206 |
| ✓ | BbhRuinNames175CGWhich of the following is a humorous edit of this artist or movie name: 'brokeback mountain'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 252 |
| ✓ | BbhRuinNames176CGWhich of the following is a humorous edit of this artist or movie name: 'godzilla vs. kong'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 483 |
| ✓ | BbhRuinNames177CGWhich of the following is a humorous edit of this artist or movie name: 'the who!'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT717 |
| ✓ | BbhRuinNames178CGWhich of the following is a humorous edit of this artist or movie name: 'the green mile'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 238 |
| ✓ | BbhRuinNames179CGWhich of the following is a humorous edit of this artist or movie name: 'the dark knight'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT913 |
| ✓ | BbhRuinNames180CGWhich of the following is a humorous edit of this artist or movie name: 'live and let die'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT546 |
| × | BbhRuinNames181CGWhich of the following is a humorous edit of this artist or movie name: 'midnight cowboy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames181CG: expected (B), got [d]
Values:
got = "d" |
Failed | IN1 004 OUT1 441 |
| ✓ | BbhRuinNames182CGWhich of the following is a humorous edit of this artist or movie name: 'snow patrol'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT827 |
| ✓ | BbhRuinNames183CGWhich of the following is a humorous edit of this artist or movie name: 'the bangles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 578 |
| ✓ | BbhRuinNames184CGWhich of the following is a humorous edit of this artist or movie name: 'state champs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 065 |
| ✓ | BbhRuinNames185CGWhich of the following is a humorous edit of this artist or movie name: 'the secret life of pets'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT965 |
| ✓ | BbhRuinNames186CGWhich of the following is a humorous edit of this artist or movie name: 'full metal jacket'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 664 |
| ✓ | BbhRuinNames187CGWhich of the following is a humorous edit of this artist or movie name: 'friday the 13th'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT1 654 |
| ✓ | BbhRuinNames188CGWhich of the following is a humorous edit of this artist or movie name: 'the bay city rollers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT767 |
| × | BbhRuinNames189CGWhich of the following is a humorous edit of this artist or movie name: 'blue'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames189CG: expected (B), got [d]
Values:
got = "d" |
Failed | IN995 OUT2 060 |
| ✓ | BbhRuinNames190CGWhich of the following is a humorous edit of this artist or movie name: 'the gap band'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT766 |
| ✓ | BbhRuinNames191CGWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT602 |
| ✓ | BbhRuinNames192CGWhich of the following is a humorous edit of this artist or movie name: 'the fall of the house of usher'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT1 123 |
| ✓ | BbhRuinNames193CGWhich of the following is a humorous edit of this artist or movie name: 'old joy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 796 |
| ✓ | BbhRuinNames194CGWhich of the following is a humorous edit of this artist or movie name: 'run dmc'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 384 |
| ✓ | BbhRuinNames195CGWhich of the following is a humorous edit of this artist or movie name: 'the grapes of wrath'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT596 |
| ✓ | BbhRuinNames196CGWhich of the following is a humorous edit of this artist or movie name: 'drag me to hell'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 390 |
| ✓ | BbhRuinNames197CGWhich of the following is a humorous edit of this artist or movie name: 'talking heads'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT953 |
| ✓ | BbhRuinNames198CGWhich of the following is a humorous edit of this artist or movie name: 'paul oakenfold'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 679 |
| ✓ | BbhRuinNames199CGWhich of the following is a humorous edit of this artist or movie name: 'brief encounter'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 124 |
| ✓ | BbhRuinNames200CGWhich of the following is a humorous edit of this artist or movie name: 'a tribe called quest'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT805 |
| × | BbhRuinNames201CGWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames201CG: expected (C), got [a]
Values:
got = "a" |
Failed | IN996 OUT1 740 |
| ✓ | BbhRuinNames202CGWhich of the following is a humorous edit of this artist or movie name: 'the wizard of oz'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 282 |
| ✓ | BbhRuinNames203CGWhich of the following is a humorous edit of this artist or movie name: 'run dmc'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT826 |
| ✓ | BbhRuinNames204CGWhich of the following is a humorous edit of this artist or movie name: 'the third man'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT3 548 |
| ✓ | BbhRuinNames205CGWhich of the following is a humorous edit of this artist or movie name: 'cat on a hot tin roof'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 102 |
| ✓ | BbhRuinNames206CGWhich of the following is a humorous edit of this artist or movie name: 'notting hill'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 034 |
| ✓ | BbhRuinNames207CGWhich of the following is a humorous edit of this artist or movie name: 'good will hunting'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT869 |
| ✓ | BbhRuinNames208CGWhich of the following is a humorous edit of this artist or movie name: 'foo fighters'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT480 |
| ✓ | BbhRuinNames209CGWhich of the following is a humorous edit of this artist or movie name: 'goo goo dolls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT772 |
Reasoning - BBH ruin names (humorous edits) (set 7)
37 passed3 failed |
92.5% |
IN40 207 OUT54 005 | |
| ✓ | BbhRuinNames210CGWhich of the following is a humorous edit of this artist or movie name: 'soundgarden'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 247 |
| ✓ | BbhRuinNames211CGWhich of the following is a humorous edit of this artist or movie name: 'peter and the test tube babies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 310 |
| ✓ | BbhRuinNames212CGWhich of the following is a humorous edit of this artist or movie name: 'blur'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 341 |
| ✓ | BbhRuinNames213CGWhich of the following is a humorous edit of this artist or movie name: 'black sabbath'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 159 |
| ✓ | BbhRuinNames214CGWhich of the following is a humorous edit of this artist or movie name: 'trolls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 956 |
| ✓ | BbhRuinNames215CGWhich of the following is a humorous edit of this artist or movie name: 'bad company'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT930 |
| ✓ | BbhRuinNames216CGWhich of the following is a humorous edit of this artist or movie name: '3:10 to yuma'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT3 917 |
| ✓ | BbhRuinNames217CGWhich of the following is a humorous edit of this artist or movie name: 'herman's hermits'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 292 |
| ✓ | BbhRuinNames218CGWhich of the following is a humorous edit of this artist or movie name: 'night of the demons'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT791 |
| ✓ | BbhRuinNames219CGWhich of the following is a humorous edit of this artist or movie name: 'the rolling stones'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT596 |
| ✓ | BbhRuinNames220CGWhich of the following is a humorous edit of this artist or movie name: 'blur'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 220 |
| ✓ | BbhRuinNames221CGWhich of the following is a humorous edit of this artist or movie name: 'a matter of life and death'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 312 |
| ✓ | BbhRuinNames222CGWhich of the following is a humorous edit of this artist or movie name: 'singin' in the rain'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT956 |
| ✓ | BbhRuinNames223CGWhich of the following is a humorous edit of this artist or movie name: 'the man with two brains'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT944 |
| × | BbhRuinNames224CGWhich of the following is a humorous edit of this artist or movie name: 'born free'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames224CG: expected (C), got [a]
Values:
got = "a" |
Failed | IN998 OUT2 023 |
| ✓ | BbhRuinNames225CGWhich of the following is a humorous edit of this artist or movie name: 'the moody blues'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 718 |
| ✓ | BbhRuinNames226CGWhich of the following is a humorous edit of this artist or movie name: 'jojo rabbit'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 851 |
| ✓ | BbhRuinNames227CGWhich of the following is a humorous edit of this artist or movie name: 'reservoir dogs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 123 |
| ✓ | BbhRuinNames228CGWhich of the following is a humorous edit of this artist or movie name: 'thunder road'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 098 |
| ✓ | BbhRuinNames229CGWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 436 |
| ✓ | BbhRuinNames230CGWhich of the following is a humorous edit of this artist or movie name: 'deadpool'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 036 |
| ✓ | BbhRuinNames231CGWhich of the following is a humorous edit of this artist or movie name: 'the lady vanishes'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 620 |
| ✓ | BbhRuinNames232CGWhich of the following is a humorous edit of this artist or movie name: 'steely dan'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 113 |
| ✓ | BbhRuinNames233CGWhich of the following is a humorous edit of this artist or movie name: 'the men who stare at goats'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 083 |
| ✓ | BbhRuinNames234CGWhich of the following is a humorous edit of this artist or movie name: 'the stone roses'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 391 |
| ✓ | BbhRuinNames235CGWhich of the following is a humorous edit of this artist or movie name: 'nine inch nails'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT468 |
| ✓ | BbhRuinNames236CGWhich of the following is a humorous edit of this artist or movie name: 'billy idol'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT734 |
| ✓ | BbhRuinNames237CGWhich of the following is a humorous edit of this artist or movie name: 'schindler's list'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT970 |
| ✓ | BbhRuinNames238CGWhich of the following is a humorous edit of this artist or movie name: 'sex pistols'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT2 649 |
| ✓ | BbhRuinNames239CGWhich of the following is a humorous edit of this artist or movie name: 'gone with the wind'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT798 |
| ✓ | BbhRuinNames240CGWhich of the following is a humorous edit of this artist or movie name: 'it's a wonderful life'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 444 |
| ✓ | BbhRuinNames241CGWhich of the following is a humorous edit of this artist or movie name: 'guns n' roses'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 003 |
| ✓ | BbhRuinNames242CGWhich of the following is a humorous edit of this artist or movie name: 'witness'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT960 |
| × | BbhRuinNames243CGWhich of the following is a humorous edit of this artist or movie name: 'killing joke'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames243CG: expected (A), got [c]
Values:
got = "c" |
Failed | IN998 OUT1 405 |
| ✓ | BbhRuinNames244CGWhich of the following is a humorous edit of this artist or movie name: 'panic! at the disco'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT564 |
| × | BbhRuinNames245CGWhich of the following is a humorous edit of this artist or movie name: 'spice girls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames245CG: expected (D), got [b]
Values:
got = "b" |
Failed | IN999 OUT1 743 |
| ✓ | BbhRuinNames246CGWhich of the following is a humorous edit of this artist or movie name: 'the police'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 417 |
| ✓ | BbhRuinNames247CGWhich of the following is a humorous edit of this artist or movie name: 'spandau ballet'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 553 |
| ✓ | BbhRuinNames248CGWhich of the following is a humorous edit of this artist or movie name: 'toto'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT2 798 |
| ✓ | BbhRuinNames249CGWhich of the following is a humorous edit of this artist or movie name: 'system of a down'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 036 |
Reasoning - BBH snarks - detect sarcasm (set 1)
27 passed5 failed |
84.4% |
IN33 394 OUT33 265 | |
| × | BbhSalientTranslation243CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSalientTranslation243CI: expected (A), got [e]
Values:
got = "e" |
Failed | IN1 202 OUT906 |
| ✓ | BbhSalientTranslation244CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 178 OUT1 950 |
| × | BbhSalientTranslation245CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSalientTranslation245CI: expected (B), got [d]
Values:
got = "d" |
Failed | IN1 192 OUT1 422 |
| ✓ | BbhSalientTranslation246CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 235 OUT989 |
| ✓ | BbhSalientTranslation247CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 190 OUT687 |
| × | BbhSalientTranslation248CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(E)")
Context: BbhSalientTranslation248CI: expected (E), got [a]
Values:
got = "a" |
Failed | IN1 185 OUT1 906 |
| × | BbhSalientTranslation249CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhSalientTranslation249CI: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 232 OUT980 |
| ✓ | BbhSnarks10CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT882 |
| ✓ | BbhSnarks11CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 575 |
| ✓ | BbhSnarks12CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 020 |
| ✓ | BbhSnarks13CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT498 |
| ✓ | BbhSnarks14CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT921 |
| ✓ | BbhSnarks15CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT983 |
| ✓ | BbhSnarks16CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT782 |
| ✓ | BbhSnarks17CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT2 129 |
| ✓ | BbhSnarks18CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT673 |
| ✓ | BbhSnarks19CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT549 |
| ✓ | BbhSnarks20CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT1 721 |
| ✓ | BbhSnarks21CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT884 |
| × | BbhSnarks22CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks22CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN994 OUT2 062 |
| ✓ | BbhSnarks23CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT1 215 |
| ✓ | BbhSnarks24CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT720 |
| ✓ | BbhSnarks25CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 175 |
| ✓ | BbhSnarks26CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT871 |
| ✓ | BbhSnarks27CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 024 |
| ✓ | BbhSnarks28CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT831 |
| ✓ | BbhSnarks29CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT702 |
| ✓ | BbhSnarks30CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT736 |
| ✓ | BbhSnarks31CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT407 |
| ✓ | BbhSnarks32CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT1 082 |
| ✓ | BbhSnarks8CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT605 |
| ✓ | BbhSnarks9CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT378 |
Reasoning - BBH snarks - detect sarcasm (set 2)
27 passed5 failed |
84.4% |
IN32 148 OUT32 696 | |
| ✓ | BbhSnarks33CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT590 |
| ✓ | BbhSnarks34CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT506 |
| ✓ | BbhSnarks35CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT935 |
| ✓ | BbhSnarks36CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT951 |
| ✓ | BbhSnarks37CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 608 |
| ✓ | BbhSnarks38CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT657 |
| ✓ | BbhSnarks39CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT653 |
| × | BbhSnarks40CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks40CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN992 OUT1 571 |
| ✓ | BbhSnarks41CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 838 |
| ✓ | BbhSnarks42CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT1 316 |
| × | BbhSnarks43CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks43CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 004 OUT649 |
| ✓ | BbhSnarks44CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT1 006 |
| ✓ | BbhSnarks45CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 371 |
| ✓ | BbhSnarks46CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT909 |
| ✓ | BbhSnarks47CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 071 |
| ✓ | BbhSnarks48CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT960 |
| × | BbhSnarks49CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSnarks49CI: expected (A), got [b]
Values:
got = "b" |
Failed | IN988 OUT1 406 |
| ✓ | BbhSnarks50CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 294 |
| × | BbhSnarks51CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks51CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 006 OUT2 034 |
| ✓ | BbhSnarks52CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT797 |
| ✓ | BbhSnarks53CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 338 |
| ✓ | BbhSnarks54CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT760 |
| ✓ | BbhSnarks55CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT914 |
| ✓ | BbhSnarks56CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT663 |
| ✓ | BbhSnarks57CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT618 |
| × | BbhSnarks58CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks58CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN996 OUT380 |
| ✓ | BbhSnarks59CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 137 |
| ✓ | BbhSnarks60CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 384 |
| ✓ | BbhSnarks61CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT782 |
| ✓ | BbhSnarks62CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT1 485 |
| ✓ | BbhSnarks63CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT478 |
| ✓ | BbhSnarks64CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT635 |
Reasoning - BBH snarks - detect sarcasm (set 3)
27 passed3 failed2 skipped |
90.0% |
IN30 091 OUT29 784 | |
| ✓ | BbhSnarks65CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT654 |
| × | BbhSnarks66CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks66CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 006 OUT900 |
| ✓ | BbhSnarks67CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT694 |
| ✓ | BbhSnarks68CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT520 |
| ✓ | BbhSnarks69CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT1 343 |
| ✓ | BbhSnarks70CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT559 |
| ✓ | BbhSnarks71CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 086 |
| ✓ | BbhSnarks72CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 776 |
| ✓ | BbhSnarks73CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT493 |
| ✓ | BbhSnarks74CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT874 |
| ✓ | BbhSnarks75CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT799 |
| ✓ | BbhSnarks76CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT693 |
| × | BbhSnarks77CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks77CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 040 OUT1 238 |
| × | BbhSnarks78CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks78CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 014 OUT887 |
| ✓ | BbhSnarks79CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT841 |
| ✓ | BbhSnarks80CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT600 |
| ✓ | BbhSnarks81CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 564 |
| ✓ | BbhSnarks82CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT1 317 |
| ✓ | BbhSnarks83CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT940 |
| ✓ | BbhSnarks84CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 033 |
| ✓ | BbhSnarks85CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT893 |
| – | BbhSnarks86CIWhich statement is sarcastic?0 AI queries
0 tool callscategory:Reasoninglevel:Hardsource:BBH Skipped: non-deterministic answer: both options are identical |
Skipped | IN0 OUT0 |
| ✓ | BbhSnarks87CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 775 |
| – | BbhSnarks88CIWhich statement is sarcastic?0 AI queries
0 tool callscategory:Reasoninglevel:Hardsource:BBH Skipped: non-deterministic answer: malformed item: single truncated option |
Skipped | IN0 OUT0 |
| ✓ | BbhSnarks89CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT971 |
| ✓ | BbhSnarks90CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT861 |
| ✓ | BbhSnarks91CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT658 |
| ✓ | BbhSnarks92CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT470 |
| ✓ | BbhSnarks93CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT2 040 |
| ✓ | BbhSnarks94CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT660 |
| ✓ | BbhSnarks95CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT701 |
| ✓ | BbhSnarks96CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT944 |
Reasoning - BBH snarks - detect sarcasm (set 4)
29 passed3 failed |
90.6% |
IN31 917 OUT30 481 | |
| ✓ | BbhSnarks100CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT561 |
| ✓ | BbhSnarks101CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 092 |
| ✓ | BbhSnarks102CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT882 |
| ✓ | BbhSnarks103CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT1 352 |
| ✓ | BbhSnarks104CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT843 |
| ✓ | BbhSnarks105CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT900 |
| ✓ | BbhSnarks106CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 056 |
| ✓ | BbhSnarks107CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT1 347 |
| ✓ | BbhSnarks108CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT929 |
| ✓ | BbhSnarks109CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT715 |
| ✓ | BbhSnarks110CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT922 |
| ✓ | BbhSnarks111CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT911 |
| ✓ | BbhSnarks112CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 391 |
| ✓ | BbhSnarks113CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT670 |
| ✓ | BbhSnarks114CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT782 |
| ✓ | BbhSnarks115CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT752 |
| ✓ | BbhSnarks116CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT908 |
| × | BbhSnarks117CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks117CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN999 OUT1 334 |
| ✓ | BbhSnarks118CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 269 |
| ✓ | BbhSnarks119CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT855 |
| ✓ | BbhSnarks120CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT712 |
| ✓ | BbhSnarks121CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT558 |
| ✓ | BbhSnarks122CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT915 |
| ✓ | BbhSnarks123CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT1 047 |
| ✓ | BbhSnarks124CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 095 |
| × | BbhSnarks125CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks125CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 000 OUT1 221 |
| ✓ | BbhSnarks126CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT363 |
| ✓ | BbhSnarks127CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT790 |
| × | BbhSnarks128CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSnarks128CI: expected (A), got [b]
Values:
got = "b" |
Failed | IN998 OUT920 |
| ✓ | BbhSnarks97CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT1 303 |
| ✓ | BbhSnarks98CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 044 |
| ✓ | BbhSnarks99CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 042 |
Reasoning - BBH snarks - detect sarcasm (set 5)
28 passed2 failed |
93.3% |
IN30 055 OUT28 330 | |
| ✓ | BbhSnarks129CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT554 |
| ✓ | BbhSnarks130CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT670 |
| ✓ | BbhSnarks131CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 004 |
| ✓ | BbhSnarks132CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT585 |
| ✓ | BbhSnarks133CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT789 |
| ✓ | BbhSnarks134CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT626 |
| ✓ | BbhSnarks135CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 036 |
| ✓ | BbhSnarks136CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT670 |
| ✓ | BbhSnarks137CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT513 |
| ✓ | BbhSnarks138CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT624 |
| ✓ | BbhSnarks139CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT705 |
| ✓ | BbhSnarks140CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT944 |
| ✓ | BbhSnarks141CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT745 |
| ✓ | BbhSnarks142CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT1 097 |
| ✓ | BbhSnarks143CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT663 |
| × | BbhSnarks144CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks144CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN998 OUT1 436 |
| ✓ | BbhSnarks145CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT799 |
| ✓ | BbhSnarks146CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT949 |
| ✓ | BbhSnarks147CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 594 |
| ✓ | BbhSnarks148CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 280 |
| ✓ | BbhSnarks149CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT494 |
| ✓ | BbhSnarks150CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 561 |
| ✓ | BbhSnarks151CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT924 |
| × | BbhSnarks152CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks152CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 016 OUT2 281 |
| ✓ | BbhSnarks153CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 217 |
| ✓ | BbhSnarks154CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT782 |
| ✓ | BbhSnarks155CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT938 |
| ✓ | BbhSnarks156CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT550 |
| ✓ | BbhSnarks157CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT788 |
| ✓ | BbhSnarks158CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 512 |
Reasoning - BBH snarks - detect sarcasm (set 6)
26 passed6 failed |
81.3% |
IN31 595 OUT31 030 | |
| ✓ | BbhSnarks159CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT619 |
| ✓ | BbhSnarks160CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 003 |
| ✓ | BbhSnarks161CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT676 |
| ✓ | BbhSnarks162CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT496 |
| ✓ | BbhSnarks163CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT641 |
| ✓ | BbhSnarks164CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 290 |
| ✓ | BbhSnarks165CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT650 |
| ✓ | BbhSnarks166CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT971 |
| ✓ | BbhSnarks167CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT594 |
| × | BbhSnarks168CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks168CJ expected (B), got [a]
Values:
got = "a" |
Failed | IN990 OUT1 302 |
| ✓ | BbhSnarks169CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT785 |
| ✓ | BbhSnarks170CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT713 |
| ✓ | BbhSnarks171CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 465 |
| ✓ | BbhSnarks172CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT767 |
| ✓ | BbhSnarks173CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT723 |
| ✓ | BbhSnarks174CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 179 |
| ✓ | BbhSnarks175CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT665 |
| ✓ | BbhSnarks176CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT739 |
| ✓ | BbhSnarks177CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT484 |
| ✓ | BbhSportsUnderstanding10CJIs the following sentence plausible? "Carson Wentz took to the ice."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN969 OUT971 |
| ✓ | BbhSportsUnderstanding11CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN972 OUT1 546 |
| × | BbhSportsUnderstanding12CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding12CJ expected yes, got [no]
Values:
got = "no" |
Failed | IN975 OUT1 542 |
| ✓ | BbhSportsUnderstanding13CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN970 OUT1 011 |
| ✓ | BbhSportsUnderstanding14CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN970 OUT971 |
| × | BbhSportsUnderstanding15CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding15CJ expected yes, got [no]
Values:
got = "no" |
Failed | IN973 OUT1 907 |
| ✓ | BbhSportsUnderstanding16CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN969 OUT784 |
| ✓ | BbhSportsUnderstanding17CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN968 OUT1 197 |
| ✓ | BbhSportsUnderstanding18CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN967 OUT1 189 |
| × | BbhSportsUnderstanding19CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding19CJ expected yes, got [no]
Values:
got = "no" |
Failed | IN974 OUT933 |
| ✓ | BbhSportsUnderstanding20CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN968 OUT914 |
| × | BbhSportsUnderstanding8CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding8CJ expected yes, got [no]
Values:
got = "no" |
Failed | IN973 OUT952 |
| × | BbhSportsUnderstanding9CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("no")
Context: BbhSportsUnderstanding9CJ expected no, got [yes]
Values:
got = "yes" |
Failed | IN969 OUT1 351 |
Reasoning - BBH temporal sequences (set 1)
31 passed1 failed |
96.9% |
IN35 127 OUT34 877 | |
| ✓ | BbhSportsUnderstanding243CKIs the following sentence plausible? "Blake Snell hit a single."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN967 OUT1 717 |
| × | BbhSportsUnderstanding244CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding244CK expected yes, got [implausible]
Values:
got = "implausible" |
Failed | IN974 OUT1 643 |
| ✓ | BbhSportsUnderstanding245CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN972 OUT572 |
| ✓ | BbhSportsUnderstanding246CKIs the following sentence plausible? "Pepe converted the first down."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN967 OUT2 039 |
| ✓ | BbhSportsUnderstanding247CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN975 OUT1 640 |
| ✓ | BbhSportsUnderstanding248CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN969 OUT971 |
| ✓ | BbhSportsUnderstanding249CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN970 OUT764 |
| ✓ | BbhTemporalSequences10CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT690 |
| ✓ | BbhTemporalSequences11CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT809 |
| ✓ | BbhTemporalSequences12CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT810 |
| ✓ | BbhTemporalSequences13CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 138 OUT1 022 |
| ✓ | BbhTemporalSequences14CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 110 |
| ✓ | BbhTemporalSequences15CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT992 |
| ✓ | BbhTemporalSequences16CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT1 164 |
| ✓ | BbhTemporalSequences17CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT847 |
| ✓ | BbhTemporalSequences18CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT1 484 |
| ✓ | BbhTemporalSequences19CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT825 |
| ✓ | BbhTemporalSequences20CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 117 |
| ✓ | BbhTemporalSequences21CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT1 206 |
| ✓ | BbhTemporalSequences22CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT1 117 |
| ✓ | BbhTemporalSequences23CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT1 073 |
| ✓ | BbhTemporalSequences24CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 175 |
| ✓ | BbhTemporalSequences25CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT1 116 |
| ✓ | BbhTemporalSequences26CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT970 |
| ✓ | BbhTemporalSequences27CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT555 |
| ✓ | BbhTemporalSequences28CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT1 560 |
| ✓ | BbhTemporalSequences29CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT963 |
| ✓ | BbhTemporalSequences30CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT539 |
| ✓ | BbhTemporalSequences31CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT1 067 |
| ✓ | BbhTemporalSequences32CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT1 125 |
| ✓ | BbhTemporalSequences8CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 139 |
| ✓ | BbhTemporalSequences9CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 056 |
Reasoning - BBH temporal sequences (set 2)
30 passed |
100.0% |
IN33 903 OUT33 434 | |
| ✓ | BbhTemporalSequences33CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT1 071 |
| ✓ | BbhTemporalSequences34CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT891 |
| ✓ | BbhTemporalSequences35CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT989 |
| ✓ | BbhTemporalSequences36CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT1 070 |
| ✓ | BbhTemporalSequences37CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT735 |
| ✓ | BbhTemporalSequences38CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT1 312 |
| ✓ | BbhTemporalSequences39CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT961 |
| ✓ | BbhTemporalSequences40CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT997 |
| ✓ | BbhTemporalSequences41CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 735 |
| ✓ | BbhTemporalSequences42CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT854 |
| ✓ | BbhTemporalSequences43CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 037 |
| ✓ | BbhTemporalSequences44CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 281 |
| ✓ | BbhTemporalSequences45CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT978 |
| ✓ | BbhTemporalSequences46CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT1 186 |
| ✓ | BbhTemporalSequences47CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT838 |
| ✓ | BbhTemporalSequences48CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 545 |
| ✓ | BbhTemporalSequences49CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT971 |
| ✓ | BbhTemporalSequences50CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 589 |
| ✓ | BbhTemporalSequences51CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT1 176 |
| ✓ | BbhTemporalSequences52CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 167 |
| ✓ | BbhTemporalSequences53CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT2 004 |
| ✓ | BbhTemporalSequences54CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT1 243 |
| ✓ | BbhTemporalSequences55CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT1 033 |
| ✓ | BbhTemporalSequences56CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT886 |
| ✓ | BbhTemporalSequences57CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT916 |
| ✓ | BbhTemporalSequences58CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT930 |
| ✓ | BbhTemporalSequences59CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT1 177 |
| ✓ | BbhTemporalSequences60CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT975 |
| ✓ | BbhTemporalSequences61CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT831 |
| ✓ | BbhTemporalSequences62CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 056 |
Reasoning - BBH temporal sequences (set 3)
32 passed |
100.0% |
IN36 111 OUT33 797 | |
| ✓ | BbhTemporalSequences63CLfind the time window when Samantha could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 083 |
| ✓ | BbhTemporalSequences64CLfind the time window when Betty could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT780 |
| ✓ | BbhTemporalSequences65CLfind the time window when William could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT1 537 |
| ✓ | BbhTemporalSequences66CLfind the time window when James could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT704 |
| ✓ | BbhTemporalSequences67CLfind the time window when Samantha could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT1 093 |
| ✓ | BbhTemporalSequences68CLfind the time window when William could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT1 160 |
| ✓ | BbhTemporalSequences69CLfind the time window when Kimberly could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT1 421 |
| ✓ | BbhTemporalSequences70CLfind the time window when Susan could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT826 |
| ✓ | BbhTemporalSequences71CLfind the time window when James could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 290 |
| ✓ | BbhTemporalSequences72CLfind the time window when Ashley could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 163 |
| ✓ | BbhTemporalSequences73CLfind the time window when Jennifer could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 024 |
| ✓ | BbhTemporalSequences74CLfind the time window when James could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT986 |
| ✓ | BbhTemporalSequences75CLfind the time window when Anthony could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT925 |
| ✓ | BbhTemporalSequences76CLfind the time window when Thomas could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 369 |
| ✓ | BbhTemporalSequences77CLfind the time window when Jennifer could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT837 |
| ✓ | BbhTemporalSequences78CLfind the time window when Sarah could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT937 |
| ✓ | BbhTemporalSequences79CLfind the time window when Susan could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT586 |
| ✓ | BbhTemporalSequences80CLfind the time window when William could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT1 131 |
| ✓ | BbhTemporalSequences81CLfind the time window when Jessica could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT832 |
| ✓ | BbhTemporalSequences82CLfind the time window when Kimberly could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT1 005 |
| ✓ | BbhTemporalSequences83CLfind the time window when Elizabeth could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 078 |
| ✓ | BbhTemporalSequences84CLfind the time window when James could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT1 440 |
| ✓ | BbhTemporalSequences85CLfind the time window when Michael could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT927 |
| ✓ | BbhTemporalSequences86CLfind the time window when Sarah could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT1 187 |
| ✓ | BbhTemporalSequences87CLfind the time window when Mark could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 109 |
| ✓ | BbhTemporalSequences88CLfind the time window when Sean could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT1 306 |
| ✓ | BbhTemporalSequences89CLfind the time window when James could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT738 |
| ✓ | BbhTemporalSequences90CLfind the time window when Jessica could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT606 |
| ✓ | BbhTemporalSequences91CLfind the time window when Tiffany could have gone to the library1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT1 214 |
| ✓ | BbhTemporalSequences92CLfind the time window when James could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT1 109 |
| ✓ | BbhTemporalSequences93CLfind the time window when Nancy could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 452 |
| ✓ | BbhTemporalSequences94CLfind the time window when Tiffany could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT942 |
Reasoning - BBH temporal sequences (set 4)
32 passed |
100.0% |
IN36 005 OUT34 282 | |
| ✓ | BbhTemporalSequences100CLfind the time window when Steven could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT1 219 |
| ✓ | BbhTemporalSequences101CLfind the time window when Jessica could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT1 173 |
| ✓ | BbhTemporalSequences102CLfind the time window when Jessica could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT920 |
| ✓ | BbhTemporalSequences103CLfind the time window when Emily could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT1 026 |
| ✓ | BbhTemporalSequences104CLfind the time window when Anthony could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT985 |
| ✓ | BbhTemporalSequences105CLfind the time window when Mark could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT724 |
| ✓ | BbhTemporalSequences106CLfind the time window when William could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 128 |
| ✓ | BbhTemporalSequences107CLfind the time window when Hannah could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 003 |
| ✓ | BbhTemporalSequences108CLfind the time window when William could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 132 |
| ✓ | BbhTemporalSequences109CLfind the time window when Nancy could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 001 |
| ✓ | BbhTemporalSequences110CLfind the time window when Andrew could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 149 |
| ✓ | BbhTemporalSequences111CLfind the time window when David could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT1 552 |
| ✓ | BbhTemporalSequences112CLfind the time window when Anthony could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT785 |
| ✓ | BbhTemporalSequences113CLfind the time window when Sarah could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT995 |
| ✓ | BbhTemporalSequences114CLfind the time window when Mary could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT920 |
| ✓ | BbhTemporalSequences115CLfind the time window when Mark could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT941 |
| ✓ | BbhTemporalSequences116CLfind the time window when William could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 447 |
| ✓ | BbhTemporalSequences117CLfind the time window when Steven could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT877 |
| ✓ | BbhTemporalSequences118CLfind the time window when Leslie could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 840 |
| ✓ | BbhTemporalSequences119CLfind the time window when Sarah could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT876 |
| ✓ | BbhTemporalSequences120CLfind the time window when Andrew could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT781 |
| ✓ | BbhTemporalSequences121CLfind the time window when David could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT1 539 |
| ✓ | BbhTemporalSequences122CLfind the time window when Jessica could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT1 179 |
| ✓ | BbhTemporalSequences123CLfind the time window when David could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT858 |
| ✓ | BbhTemporalSequences124CLfind the time window when Jessica could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT928 |
| ✓ | BbhTemporalSequences125CLfind the time window when Mary could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT958 |
| ✓ | BbhTemporalSequences126CLfind the time window when Michael could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT818 |
| ✓ | BbhTemporalSequences95CLfind the time window when Jennifer could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT861 |
| ✓ | BbhTemporalSequences96CLfind the time window when Sean could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 141 OUT884 |
| ✓ | BbhTemporalSequences97CLfind the time window when Samantha could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 034 |
| ✓ | BbhTemporalSequences98CLfind the time window when Ashley could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 350 |
| ✓ | BbhTemporalSequences99CLfind the time window when Sarah could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 137 OUT1 399 |
Reasoning - BBH temporal sequences (set 5)
32 passed |
100.0% |
IN36 149 OUT33 825 | |
| ✓ | BbhTemporalSequences127CLfind the time window when David could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 443 |
| ✓ | BbhTemporalSequences128CLfind the time window when Andrew could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT955 |
| ✓ | BbhTemporalSequences129CLfind the time window when Leslie could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT645 |
| ✓ | BbhTemporalSequences130CLfind the time window when Mary could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT945 |
| ✓ | BbhTemporalSequences131CLfind the time window when Ashley could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT1 278 |
| ✓ | BbhTemporalSequences132CLfind the time window when Samantha could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT871 |
| ✓ | BbhTemporalSequences133CLfind the time window when Elizabeth could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT1 726 |
| ✓ | BbhTemporalSequences134CLfind the time window when Sean could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT953 |
| ✓ | BbhTemporalSequences135CLfind the time window when Elizabeth could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT1 061 |
| ✓ | BbhTemporalSequences136CLfind the time window when Michael could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 317 |
| ✓ | BbhTemporalSequences137CLfind the time window when Andrew could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 030 |
| ✓ | BbhTemporalSequences138CLfind the time window when Nancy could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 159 |
| ✓ | BbhTemporalSequences139CLfind the time window when Jason could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 123 OUT829 |
| ✓ | BbhTemporalSequences140CLfind the time window when Anthony could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 103 |
| ✓ | BbhTemporalSequences141CLfind the time window when Leslie could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT682 |
| ✓ | BbhTemporalSequences142CLfind the time window when David could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT1 301 |
| ✓ | BbhTemporalSequences143CLfind the time window when Hannah could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT1 333 |
| ✓ | BbhTemporalSequences144CLfind the time window when Sarah could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT898 |
| ✓ | BbhTemporalSequences145CLfind the time window when Andrew could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT1 014 |
| ✓ | BbhTemporalSequences146CLfind the time window when William could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 251 |
| ✓ | BbhTemporalSequences147CLfind the time window when Elizabeth could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT788 |
| ✓ | BbhTemporalSequences148CLfind the time window when Sarah could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 117 |
| ✓ | BbhTemporalSequences149CLfind the time window when David could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT794 |
| ✓ | BbhTemporalSequences150CLfind the time window when Jason could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 078 |
| ✓ | BbhTemporalSequences151CLfind the time window when Anthony could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT706 |
| ✓ | BbhTemporalSequences152CLfind the time window when Sarah could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT1 089 |
| ✓ | BbhTemporalSequences153CLfind the time window when Hannah could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT1 442 |
| ✓ | BbhTemporalSequences154CLfind the time window when Mark could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT861 |
| ✓ | BbhTemporalSequences155CLfind the time window when Sarah could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 316 |
| ✓ | BbhTemporalSequences156CLfind the time window when James could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT1 220 |
| ✓ | BbhTemporalSequences157CLfind the time window when Emily could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT673 |
| ✓ | BbhTemporalSequences158CLfind the time window when David could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT947 |
Reasoning - BBH temporal sequences (set 6)
32 passed |
100.0% |
IN36 117 OUT33 678 | |
| ✓ | BbhTemporalSequences159CLfind the time window when Jennifer could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT1 221 |
| ✓ | BbhTemporalSequences160CLfind the time window when Ashley could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT1 128 |
| ✓ | BbhTemporalSequences161CLfind the time window when William could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 123 OUT1 345 |
| ✓ | BbhTemporalSequences162CLfind the time window when Jennifer could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT824 |
| ✓ | BbhTemporalSequences163CLfind the time window when Jennifer could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT590 |
| ✓ | BbhTemporalSequences164CLfind the time window when Jennifer could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT862 |
| ✓ | BbhTemporalSequences165CLfind the time window when Nancy could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT1 002 |
| ✓ | BbhTemporalSequences166CLfind the time window when Samantha could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 366 |
| ✓ | BbhTemporalSequences167CLfind the time window when Tiffany could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT994 |
| ✓ | BbhTemporalSequences168CLfind the time window when Leslie could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT994 |
| ✓ | BbhTemporalSequences169CLfind the time window when Hannah could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT1 523 |
| ✓ | BbhTemporalSequences170CLfind the time window when Anthony could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT1 325 |
| ✓ | BbhTemporalSequences171CLfind the time window when Hannah could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT1 306 |
| ✓ | BbhTemporalSequences172CLfind the time window when Leslie could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 391 |
| ✓ | BbhTemporalSequences173CLfind the time window when Anthony could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT986 |
| ✓ | BbhTemporalSequences174CLfind the time window when Thomas could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT653 |
| ✓ | BbhTemporalSequences175CLfind the time window when Jason could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT820 |
| ✓ | BbhTemporalSequences176CLfind the time window when Betty could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT764 |
| ✓ | BbhTemporalSequences177CLfind the time window when Ashley could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 459 |
| ✓ | BbhTemporalSequences178CLfind the time window when David could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT1 238 |
| ✓ | BbhTemporalSequences179CLfind the time window when Leslie could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 273 |
| ✓ | BbhTemporalSequences180CLfind the time window when Ashley could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT726 |
| ✓ | BbhTemporalSequences181CLfind the time window when Michael could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT1 098 |
| ✓ | BbhTemporalSequences182CLfind the time window when Jason could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 044 |
| ✓ | BbhTemporalSequences183CLfind the time window when Sean could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT1 022 |
| ✓ | BbhTemporalSequences184CLfind the time window when Sarah could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT638 |
| ✓ | BbhTemporalSequences185CLfind the time window when Andrew could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 132 |
| ✓ | BbhTemporalSequences186CLfind the time window when James could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT954 |
| ✓ | BbhTemporalSequences187CLfind the time window when Ashley could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT858 |
| ✓ | BbhTemporalSequences188CLfind the time window when Ashley could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT872 |
| ✓ | BbhTemporalSequences189CLfind the time window when Sarah could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT838 |
| ✓ | BbhTemporalSequences190CLfind the time window when Michael could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT1 432 |
Reasoning - BBH temporal sequences (set 7)
30 passed |
100.0% |
IN33 806 OUT33 738 | |
| ✓ | BbhTemporalSequences191CLfind the time window when Anthony could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 372 |
| ✓ | BbhTemporalSequences192CLfind the time window when William could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 637 |
| ✓ | BbhTemporalSequences193CLfind the time window when Susan could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT1 094 |
| ✓ | BbhTemporalSequences194CLfind the time window when Sarah could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT700 |
| ✓ | BbhTemporalSequences195CLfind the time window when Emily could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT854 |
| ✓ | BbhTemporalSequences196CLfind the time window when Emily could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT1 148 |
| ✓ | BbhTemporalSequences197CLfind the time window when Sean could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 152 |
| ✓ | BbhTemporalSequences198CLfind the time window when Elizabeth could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 141 OUT1 061 |
| ✓ | BbhTemporalSequences199CLfind the time window when William could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 491 |
| ✓ | BbhTemporalSequences200CLfind the time window when Thomas could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT1 194 |
| ✓ | BbhTemporalSequences201CLfind the time window when Susan could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT844 |
| ✓ | BbhTemporalSequences202CLfind the time window when Kimberly could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 158 OUT1 201 |
| ✓ | BbhTemporalSequences203CLfind the time window when Leslie could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT888 |
| ✓ | BbhTemporalSequences204CLfind the time window when William could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT793 |
| ✓ | BbhTemporalSequences205CLfind the time window when Andrew could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT1 458 |
| ✓ | BbhTemporalSequences206CLfind the time window when Jennifer could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT980 |
| ✓ | BbhTemporalSequences207CLfind the time window when Susan could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 426 |
| ✓ | BbhTemporalSequences208CLfind the time window when James could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT1 017 |
| ✓ | BbhTemporalSequences209CLfind the time window when Tiffany could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 042 |
| ✓ | BbhTemporalSequences210CLfind the time window when Andrew could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 171 |
| ✓ | BbhTemporalSequences211CLfind the time window when Thomas could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT692 |
| ✓ | BbhTemporalSequences212CLfind the time window when David could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT1 189 |
| ✓ | BbhTemporalSequences213CLfind the time window when Elizabeth could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 039 |
| ✓ | BbhTemporalSequences214CLfind the time window when Jason could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT1 152 |
| ✓ | BbhTemporalSequences215CLfind the time window when Sarah could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT905 |
| ✓ | BbhTemporalSequences216CLfind the time window when William could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 072 |
| ✓ | BbhTemporalSequences217CLfind the time window when Ashley could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT961 |
| ✓ | BbhTemporalSequences218CLfind the time window when William could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT1 754 |
| ✓ | BbhTemporalSequences219CLfind the time window when Sarah could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT1 149 |
| ✓ | BbhTemporalSequences220CLfind the time window when Nancy could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 302 |
Reasoning - BBH temporal sequences (set 8)
29 passed |
100.0% |
IN32 959 OUT33 079 | |
| ✓ | BbhTemporalSequences221CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT927 |
| ✓ | BbhTemporalSequences222CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT1 698 |
| ✓ | BbhTemporalSequences223CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT1 168 |
| ✓ | BbhTemporalSequences224CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT1 174 |
| ✓ | BbhTemporalSequences225CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 312 |
| ✓ | BbhTemporalSequences226CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT1 340 |
| ✓ | BbhTemporalSequences227CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT1 153 |
| ✓ | BbhTemporalSequences228CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT1 684 |
| ✓ | BbhTemporalSequences229CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 121 OUT940 |
| ✓ | BbhTemporalSequences230CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT1 188 |
| ✓ | BbhTemporalSequences231CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 319 |
| ✓ | BbhTemporalSequences232CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT1 723 |
| ✓ | BbhTemporalSequences233CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 033 |
| ✓ | BbhTemporalSequences234CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 006 |
| ✓ | BbhTemporalSequences235CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT1 028 |
| ✓ | BbhTemporalSequences236CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT1 291 |
| ✓ | BbhTemporalSequences237CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT903 |
| ✓ | BbhTemporalSequences238CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT770 |
| ✓ | BbhTemporalSequences239CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 031 |
| ✓ | BbhTemporalSequences240CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT1 033 |
| ✓ | BbhTemporalSequences241CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT1 158 |
| ✓ | BbhTemporalSequences242CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT958 |
| ✓ | BbhTemporalSequences243CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT832 |
| ✓ | BbhTemporalSequences244CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT1 194 |
| ✓ | BbhTemporalSequences245CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 030 |
| ✓ | BbhTemporalSequences246CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT887 |
| ✓ | BbhTemporalSequences247CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT651 |
| ✓ | BbhTemporalSequences248CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT1 330 |
| ✓ | BbhTemporalSequences249CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT1 318 |
Reasoning - BBH tracking shuffled objects (3) (set 1)
30 passed |
100.0% |
IN31 987 OUT27 859 | |
| ✓ | BbhTrackingThree10CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT482 |
| ✓ | BbhTrackingThree11CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT975 |
| ✓ | BbhTrackingThree12CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT689 |
| ✓ | BbhTrackingThree13CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT931 |
| ✓ | BbhTrackingThree14CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 038 |
| ✓ | BbhTrackingThree15CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT689 |
| ✓ | BbhTrackingThree16CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT827 |
| ✓ | BbhTrackingThree17CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT823 |
| ✓ | BbhTrackingThree18CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT1 244 |
| ✓ | BbhTrackingThree19CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 235 |
| ✓ | BbhTrackingThree20CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT1 179 |
| ✓ | BbhTrackingThree21CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT921 |
| ✓ | BbhTrackingThree22CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT950 |
| ✓ | BbhTrackingThree23CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT656 |
| ✓ | BbhTrackingThree24CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT737 |
| ✓ | BbhTrackingThree25CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT933 |
| ✓ | BbhTrackingThree26CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT674 |
| ✓ | BbhTrackingThree27CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT682 |
| ✓ | BbhTrackingThree28CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT970 |
| ✓ | BbhTrackingThree29CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 182 |
| ✓ | BbhTrackingThree30CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT662 |
| ✓ | BbhTrackingThree31CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 861 |
| ✓ | BbhTrackingThree32CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 092 |
| ✓ | BbhTrackingThree33CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT889 |
| ✓ | BbhTrackingThree34CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT1 038 |
| ✓ | BbhTrackingThree35CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT932 |
| ✓ | BbhTrackingThree36CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 306 |
| ✓ | BbhTrackingThree37CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT647 |
| ✓ | BbhTrackingThree8CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 021 |
| ✓ | BbhTrackingThree9CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT594 |
Reasoning - BBH tracking shuffled objects (3) (set 2)
30 passed |
100.0% |
IN31 998 OUT25 098 | |
| ✓ | BbhTrackingThree38CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT663 |
| ✓ | BbhTrackingThree39CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 446 |
| ✓ | BbhTrackingThree40CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT668 |
| ✓ | BbhTrackingThree41CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT568 |
| ✓ | BbhTrackingThree42CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT849 |
| ✓ | BbhTrackingThree43CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT530 |
| ✓ | BbhTrackingThree44CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT1 424 |
| ✓ | BbhTrackingThree45CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT890 |
| ✓ | BbhTrackingThree46CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT702 |
| ✓ | BbhTrackingThree47CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT604 |
| ✓ | BbhTrackingThree48CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT904 |
| ✓ | BbhTrackingThree49CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT847 |
| ✓ | BbhTrackingThree50CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT875 |
| ✓ | BbhTrackingThree51CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT890 |
| ✓ | BbhTrackingThree52CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT779 |
| ✓ | BbhTrackingThree53CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT988 |
| ✓ | BbhTrackingThree54CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT991 |
| ✓ | BbhTrackingThree55CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT673 |
| ✓ | BbhTrackingThree56CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT666 |
| ✓ | BbhTrackingThree57CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT730 |
| ✓ | BbhTrackingThree58CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT835 |
| ✓ | BbhTrackingThree59CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT648 |
| ✓ | BbhTrackingThree60CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT1 104 |
| ✓ | BbhTrackingThree61CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 034 |
| ✓ | BbhTrackingThree62CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT563 |
| ✓ | BbhTrackingThree63CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT475 |
| ✓ | BbhTrackingThree64CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT869 |
| ✓ | BbhTrackingThree65CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT863 |
| ✓ | BbhTrackingThree66CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT778 |
| ✓ | BbhTrackingThree67CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 242 |
Reasoning - BBH tracking shuffled objects (3) (set 3)
30 passed |
100.0% |
IN32 033 OUT23 889 | |
| ✓ | BbhTrackingThree68CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT959 |
| ✓ | BbhTrackingThree69CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT807 |
| ✓ | BbhTrackingThree70CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT952 |
| ✓ | BbhTrackingThree71CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT607 |
| ✓ | BbhTrackingThree72CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT881 |
| ✓ | BbhTrackingThree73CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT755 |
| ✓ | BbhTrackingThree74CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT711 |
| ✓ | BbhTrackingThree75CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT701 |
| ✓ | BbhTrackingThree76CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT939 |
| ✓ | BbhTrackingThree77CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT736 |
| ✓ | BbhTrackingThree78CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 153 |
| ✓ | BbhTrackingThree79CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT751 |
| ✓ | BbhTrackingThree80CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT1 181 |
| ✓ | BbhTrackingThree81CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT724 |
| ✓ | BbhTrackingThree82CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT818 |
| ✓ | BbhTrackingThree83CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT422 |
| ✓ | BbhTrackingThree84CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT717 |
| ✓ | BbhTrackingThree85CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT883 |
| ✓ | BbhTrackingThree86CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT760 |
| ✓ | BbhTrackingThree87CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT636 |
| ✓ | BbhTrackingThree88CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 201 |
| ✓ | BbhTrackingThree89CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT698 |
| ✓ | BbhTrackingThree90CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT527 |
| ✓ | BbhTrackingThree91CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT715 |
| ✓ | BbhTrackingThree92CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT835 |
| ✓ | BbhTrackingThree93CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT914 |
| ✓ | BbhTrackingThree94CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT910 |
| ✓ | BbhTrackingThree95CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT652 |
| ✓ | BbhTrackingThree96CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT745 |
| ✓ | BbhTrackingThree97CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT599 |
Reasoning - BBH tracking shuffled objects (3) (set 4)
29 passed |
100.0% |
IN30 903 OUT24 842 | |
| ✓ | BbhTrackingThree100CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT974 |
| ✓ | BbhTrackingThree101CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 160 |
| ✓ | BbhTrackingThree102CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 175 |
| ✓ | BbhTrackingThree103CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT739 |
| ✓ | BbhTrackingThree104CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT698 |
| ✓ | BbhTrackingThree105CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 429 |
| ✓ | BbhTrackingThree106CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT571 |
| ✓ | BbhTrackingThree107CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT672 |
| ✓ | BbhTrackingThree108CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT676 |
| ✓ | BbhTrackingThree109CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 523 |
| ✓ | BbhTrackingThree110CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT832 |
| ✓ | BbhTrackingThree111CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT553 |
| ✓ | BbhTrackingThree112CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT669 |
| ✓ | BbhTrackingThree113CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT898 |
| ✓ | BbhTrackingThree114CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT824 |
| ✓ | BbhTrackingThree115CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 017 |
| ✓ | BbhTrackingThree116CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT572 |
| ✓ | BbhTrackingThree117CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT792 |
| ✓ | BbhTrackingThree118CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 249 |
| ✓ | BbhTrackingThree119CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT757 |
| ✓ | BbhTrackingThree120CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT809 |
| ✓ | BbhTrackingThree121CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT791 |
| ✓ | BbhTrackingThree122CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT858 |
| ✓ | BbhTrackingThree123CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT575 |
| ✓ | BbhTrackingThree124CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT752 |
| ✓ | BbhTrackingThree125CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT905 |
| ✓ | BbhTrackingThree126CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT687 |
| ✓ | BbhTrackingThree98CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT671 |
| ✓ | BbhTrackingThree99CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 014 |
Reasoning - BBH tracking shuffled objects (3) (set 5)
32 passed |
100.0% |
IN34 179 OUT27 113 | |
| ✓ | BbhTrackThree127CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT643 |
| ✓ | BbhTrackThree128CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT596 |
| ✓ | BbhTrackThree129CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT800 |
| ✓ | BbhTrackThree130CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT841 |
| ✓ | BbhTrackThree131CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT1 277 |
| ✓ | BbhTrackThree132CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT841 |
| ✓ | BbhTrackThree133CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT961 |
| ✓ | BbhTrackThree134CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT683 |
| ✓ | BbhTrackThree135CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT817 |
| ✓ | BbhTrackThree136CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT829 |
| ✓ | BbhTrackThree137CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 585 |
| ✓ | BbhTrackThree138CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT898 |
| ✓ | BbhTrackThree139CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT738 |
| ✓ | BbhTrackThree140CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT778 |
| ✓ | BbhTrackThree141CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT627 |
| ✓ | BbhTrackThree142CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT676 |
| ✓ | BbhTrackThree143CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT892 |
| ✓ | BbhTrackThree144CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT636 |
| ✓ | BbhTrackThree145CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT597 |
| ✓ | BbhTrackThree146CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT783 |
| ✓ | BbhTrackThree147CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT643 |
| ✓ | BbhTrackThree148CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT705 |
| ✓ | BbhTrackThree149CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT621 |
| ✓ | BbhTrackThree150CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 039 |
| ✓ | BbhTrackThree151CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT901 |
| ✓ | BbhTrackThree152CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 006 |
| ✓ | BbhTrackThree153CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT889 |
| ✓ | BbhTrackThree154CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT809 |
| ✓ | BbhTrackThree155CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 196 |
| ✓ | BbhTrackThree156CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 266 |
| ✓ | BbhTrackThree157CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT837 |
| ✓ | BbhTrackThree158CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT703 |
Reasoning - BBH tracking shuffled objects (3) (set 6)
32 passed |
100.0% |
IN34 087 OUT28 280 | |
| ✓ | BbhTrackThree159CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT648 |
| ✓ | BbhTrackThree160CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT842 |
| ✓ | BbhTrackThree161CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT767 |
| ✓ | BbhTrackThree162CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT619 |
| ✓ | BbhTrackThree163CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT939 |
| ✓ | BbhTrackThree164CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 123 |
| ✓ | BbhTrackThree165CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT563 |
| ✓ | BbhTrackThree166CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT664 |
| ✓ | BbhTrackThree167CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT781 |
| ✓ | BbhTrackThree168CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT885 |
| ✓ | BbhTrackThree169CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 257 |
| ✓ | BbhTrackThree170CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT610 |
| ✓ | BbhTrackThree171CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT914 |
| ✓ | BbhTrackThree172CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 034 |
| ✓ | BbhTrackThree173CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT621 |
| ✓ | BbhTrackThree174CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT827 |
| ✓ | BbhTrackThree175CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT2 153 |
| ✓ | BbhTrackThree176CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 084 |
| ✓ | BbhTrackThree177CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT766 |
| ✓ | BbhTrackThree178CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT653 |
| ✓ | BbhTrackThree179CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT967 |
| ✓ | BbhTrackThree180CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT764 |
| ✓ | BbhTrackThree181CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT999 |
| ✓ | BbhTrackThree182CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT658 |
| ✓ | BbhTrackThree183CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT981 |
| ✓ | BbhTrackThree184CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 074 |
| ✓ | BbhTrackThree185CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT815 |
| ✓ | BbhTrackThree186CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 171 |
| ✓ | BbhTrackThree187CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT910 |
| ✓ | BbhTrackThree188CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT832 |
| ✓ | BbhTrackThree189CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT837 |
| ✓ | BbhTrackThree190CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT522 |
Reasoning - BBH tracking shuffled objects (3) (set 7)
32 passed |
100.0% |
IN34 106 OUT28 327 | |
| ✓ | BbhTrackThree191CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT928 |
| ✓ | BbhTrackThree192CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT679 |
| ✓ | BbhTrackThree193CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT916 |
| ✓ | BbhTrackThree194CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT935 |
| ✓ | BbhTrackThree195CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT891 |
| ✓ | BbhTrackThree196CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT983 |
| ✓ | BbhTrackThree197CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT868 |
| ✓ | BbhTrackThree198CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT956 |
| ✓ | BbhTrackThree199CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT952 |
| ✓ | BbhTrackThree200CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT959 |
| ✓ | BbhTrackThree201CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT513 |
| ✓ | BbhTrackThree202CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT735 |
| ✓ | BbhTrackThree203CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT881 |
| ✓ | BbhTrackThree204CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT751 |
| ✓ | BbhTrackThree205CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT771 |
| ✓ | BbhTrackThree206CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT825 |
| ✓ | BbhTrackThree207CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT776 |
| ✓ | BbhTrackThree208CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT829 |
| ✓ | BbhTrackThree209CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT626 |
| ✓ | BbhTrackThree210CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT990 |
| ✓ | BbhTrackThree211CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT820 |
| ✓ | BbhTrackThree212CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT962 |
| ✓ | BbhTrackThree213CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT699 |
| ✓ | BbhTrackThree214CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT684 |
| ✓ | BbhTrackThree215CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 115 |
| ✓ | BbhTrackThree216CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT695 |
| ✓ | BbhTrackThree217CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT1 544 |
| ✓ | BbhTrackThree218CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 549 |
| ✓ | BbhTrackThree219CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 071 |
| ✓ | BbhTrackThree220CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 073 |
| ✓ | BbhTrackThree221CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT551 |
| ✓ | BbhTrackThree222CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT800 |
Reasoning - BBH tracking shuffled objects (3) (set 8)
27 passed |
100.0% |
IN28 780 OUT24 765 | |
| ✓ | BbhTrackThree223CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT644 |
| ✓ | BbhTrackThree224CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 000 |
| ✓ | BbhTrackThree225CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 724 |
| ✓ | BbhTrackThree226CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT802 |
| ✓ | BbhTrackThree227CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT899 |
| ✓ | BbhTrackThree228CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT791 |
| ✓ | BbhTrackThree229CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT993 |
| ✓ | BbhTrackThree230CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 047 |
| ✓ | BbhTrackThree231CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT702 |
| ✓ | BbhTrackThree232CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT481 |
| ✓ | BbhTrackThree233CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 225 |
| ✓ | BbhTrackThree234CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT456 |
| ✓ | BbhTrackThree235CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT1 037 |
| ✓ | BbhTrackThree236CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT686 |
| ✓ | BbhTrackThree237CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 448 |
| ✓ | BbhTrackThree238CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT617 |
| ✓ | BbhTrackThree239CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 247 |
| ✓ | BbhTrackThree240CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT831 |
| ✓ | BbhTrackThree241CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 030 |
| ✓ | BbhTrackThree242CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT589 |
| ✓ | BbhTrackThree243CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT657 |
| ✓ | BbhTrackThree244CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT942 |
| ✓ | BbhTrackThree245CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT698 |
| ✓ | BbhTrackThree246CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT818 |
| ✓ | BbhTrackThree247CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT1 500 |
| ✓ | BbhTrackThree248CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 136 |
| ✓ | BbhTrackThree249CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT765 |
Reasoning - BBH tracking shuffled objects (5) (set 1)
33 passed |
100.0% |
IN36 651 OUT43 842 | |
| ✓ | BbhTrackingShuffledFive10CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 100 |
| ✓ | BbhTrackingShuffledFive11CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT919 |
| ✓ | BbhTrackingShuffledFive12CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 417 |
| ✓ | BbhTrackingShuffledFive13CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 013 |
| ✓ | BbhTrackingShuffledFive14CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 772 |
| ✓ | BbhTrackingShuffledFive15CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT893 |
| ✓ | BbhTrackingShuffledFive16CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT950 |
| ✓ | BbhTrackingShuffledFive17CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 639 |
| ✓ | BbhTrackingShuffledFive18CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 223 |
| ✓ | BbhTrackingShuffledFive19CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 014 |
| ✓ | BbhTrackingShuffledFive20CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 394 |
| ✓ | BbhTrackingShuffledFive21CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 441 |
| ✓ | BbhTrackingShuffledFive22CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 388 |
| ✓ | BbhTrackingShuffledFive23CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 161 |
| ✓ | BbhTrackingShuffledFive24CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT3 275 |
| ✓ | BbhTrackingShuffledFive25CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 280 |
| ✓ | BbhTrackingShuffledFive26CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT963 |
| ✓ | BbhTrackingShuffledFive27CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 797 |
| ✓ | BbhTrackingShuffledFive28CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 221 |
| ✓ | BbhTrackingShuffledFive29CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 366 |
| ✓ | BbhTrackingShuffledFive30CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 126 |
| ✓ | BbhTrackingShuffledFive31CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 268 |
| ✓ | BbhTrackingShuffledFive32CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 125 |
| ✓ | BbhTrackingShuffledFive33CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 362 |
| ✓ | BbhTrackingShuffledFive34CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 943 |
| ✓ | BbhTrackingShuffledFive35CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 400 |
| ✓ | BbhTrackingShuffledFive36CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 029 |
| ✓ | BbhTrackingShuffledFive37CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT1 265 |
| ✓ | BbhTrackingShuffledFive38CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 361 |
| ✓ | BbhTrackingShuffledFive39CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 130 |
| ✓ | BbhTrackingShuffledFive40CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 246 |
| ✓ | BbhTrackingShuffledFive8CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 113 |
| ✓ | BbhTrackingShuffledFive9CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 248 |
Reasoning - BBH tracking shuffled objects (5) (set 2)
33 passed |
100.0% |
IN36 557 OUT46 159 | |
| ✓ | BbhTrackingShuffledFive41CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 335 |
| ✓ | BbhTrackingShuffledFive42CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 265 |
| ✓ | BbhTrackingShuffledFive43CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 075 |
| ✓ | BbhTrackingShuffledFive44CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 599 |
| ✓ | BbhTrackingShuffledFive45CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 504 |
| ✓ | BbhTrackingShuffledFive46CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 937 |
| ✓ | BbhTrackingShuffledFive47CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 296 |
| ✓ | BbhTrackingShuffledFive48CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT2 001 |
| ✓ | BbhTrackingShuffledFive49CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 157 |
| ✓ | BbhTrackingShuffledFive50CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 267 |
| ✓ | BbhTrackingShuffledFive51CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 258 |
| ✓ | BbhTrackingShuffledFive52CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT973 |
| ✓ | BbhTrackingShuffledFive53CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 369 |
| ✓ | BbhTrackingShuffledFive54CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 488 |
| ✓ | BbhTrackingShuffledFive55CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT995 |
| ✓ | BbhTrackingShuffledFive56CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 119 |
| ✓ | BbhTrackingShuffledFive57CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 167 |
| ✓ | BbhTrackingShuffledFive58CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 403 |
| ✓ | BbhTrackingShuffledFive59CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT957 |
| ✓ | BbhTrackingShuffledFive60CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 600 |
| ✓ | BbhTrackingShuffledFive61CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 460 |
| ✓ | BbhTrackingShuffledFive62CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 655 |
| ✓ | BbhTrackingShuffledFive63CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 885 |
| ✓ | BbhTrackingShuffledFive64CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT961 |
| ✓ | BbhTrackingShuffledFive65CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 404 |
| ✓ | BbhTrackingShuffledFive66CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 517 |
| ✓ | BbhTrackingShuffledFive67CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 662 |
| ✓ | BbhTrackingShuffledFive68CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 879 |
| ✓ | BbhTrackingShuffledFive69CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT932 |
| ✓ | BbhTrackingShuffledFive70CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 383 |
| ✓ | BbhTrackingShuffledFive71CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 562 |
| ✓ | BbhTrackingShuffledFive72CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 485 |
| ✓ | BbhTrackingShuffledFive73CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 609 |
Reasoning - BBH tracking shuffled objects (5) (set 3)
32 passed |
100.0% |
IN35 526 OUT43 615 | |
| ✓ | BbhTrackingShuffledFive100CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 136 |
| ✓ | BbhTrackingShuffledFive101CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 342 |
| ✓ | BbhTrackingShuffledFive102CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 392 |
| ✓ | BbhTrackingShuffledFive103CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 359 |
| ✓ | BbhTrackingShuffledFive104CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 277 |
| ✓ | BbhTrackingShuffledFive105CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT918 |
| ✓ | BbhTrackingShuffledFive74CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 295 |
| ✓ | BbhTrackingShuffledFive75CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 267 |
| ✓ | BbhTrackingShuffledFive76CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT941 |
| ✓ | BbhTrackingShuffledFive77CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 052 |
| ✓ | BbhTrackingShuffledFive78CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 562 |
| ✓ | BbhTrackingShuffledFive79CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT1 292 |
| ✓ | BbhTrackingShuffledFive80CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 528 |
| ✓ | BbhTrackingShuffledFive81CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 820 |
| ✓ | BbhTrackingShuffledFive82CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 504 |
| ✓ | BbhTrackingShuffledFive83CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 060 |
| ✓ | BbhTrackingShuffledFive84CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT1 330 |
| ✓ | BbhTrackingShuffledFive85CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT991 |
| ✓ | BbhTrackingShuffledFive86CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 573 |
| ✓ | BbhTrackingShuffledFive87CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 100 |
| ✓ | BbhTrackingShuffledFive88CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 274 |
| ✓ | BbhTrackingShuffledFive89CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 934 |
| ✓ | BbhTrackingShuffledFive90CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 311 |
| ✓ | BbhTrackingShuffledFive91CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 995 |
| ✓ | BbhTrackingShuffledFive92CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 031 |
| ✓ | BbhTrackingShuffledFive93CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 380 |
| ✓ | BbhTrackingShuffledFive94CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 186 |
| ✓ | BbhTrackingShuffledFive95CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 671 |
| ✓ | BbhTrackingShuffledFive96CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 251 |
| ✓ | BbhTrackingShuffledFive97CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 709 |
| ✓ | BbhTrackingShuffledFive98CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 916 |
| ✓ | BbhTrackingShuffledFive99CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 218 |
Reasoning - BBH tracking shuffled objects (5) (set 4)
31 passed |
100.0% |
IN34 429 OUT41 703 | |
| ✓ | BbhTrackingShuffledFive106CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 088 |
| ✓ | BbhTrackingShuffledFive107CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 847 |
| ✓ | BbhTrackingShuffledFive108CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 051 |
| ✓ | BbhTrackingShuffledFive109CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 128 |
| ✓ | BbhTrackingShuffledFive110CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 145 |
| ✓ | BbhTrackingShuffledFive111CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 194 |
| ✓ | BbhTrackingShuffledFive112CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 889 |
| ✓ | BbhTrackingShuffledFive113CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT963 |
| ✓ | BbhTrackingShuffledFive114CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 262 |
| ✓ | BbhTrackingShuffledFive115CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT1 364 |
| ✓ | BbhTrackingShuffledFive116CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT1 307 |
| ✓ | BbhTrackingShuffledFive117CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT1 359 |
| ✓ | BbhTrackingShuffledFive118CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 400 |
| ✓ | BbhTrackingShuffledFive119CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 161 |
| ✓ | BbhTrackingShuffledFive120CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 279 |
| ✓ | BbhTrackingShuffledFive121CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 018 |
| ✓ | BbhTrackingShuffledFive122CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 710 |
| ✓ | BbhTrackingShuffledFive123CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 801 |
| ✓ | BbhTrackingShuffledFive124CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 074 |
| ✓ | BbhTrackingShuffledFive125CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 148 |
| ✓ | BbhTrackingShuffledFive126CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 418 |
| ✓ | BbhTrackingShuffledFive127CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 849 |
| ✓ | BbhTrackingShuffledFive128CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 293 |
| ✓ | BbhTrackingShuffledFive129CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 233 |
| ✓ | BbhTrackingShuffledFive130CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT921 |
| ✓ | BbhTrackingShuffledFive131CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 103 |
| ✓ | BbhTrackingShuffledFive132CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 752 |
| ✓ | BbhTrackingShuffledFive133CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 335 |
| ✓ | BbhTrackingShuffledFive134CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT970 |
| ✓ | BbhTrackingShuffledFive135CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 605 |
| ✓ | BbhTrackingShuffledFive136CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 036 |
Reasoning - BBH tracking shuffled objects (5) (set 5)
30 passed2 failed |
93.8% |
IN35 499 OUT47 180 | |
| ✓ | BbhTrackFive137CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 494 |
| ✓ | BbhTrackFive138CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 301 |
| ✓ | BbhTrackFive139CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT2 585 |
| ✓ | BbhTrackFive140CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 542 |
| ✓ | BbhTrackFive141CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 047 |
| ✓ | BbhTrackFive142CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 042 |
| ✓ | BbhTrackFive143CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 299 |
| ✓ | BbhTrackFive144CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 060 |
| ✓ | BbhTrackFive145CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 741 |
| ✓ | BbhTrackFive146CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT1 296 |
| ✓ | BbhTrackFive147CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 610 |
| ✓ | BbhTrackFive148CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 451 |
| ✓ | BbhTrackFive149CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 247 |
| ✓ | BbhTrackFive150CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 082 |
| ✓ | BbhTrackFive151CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 373 |
| ✓ | BbhTrackFive152CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 518 |
| ✓ | BbhTrackFive153CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 271 |
| ✓ | BbhTrackFive154CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT2 201 |
| ✓ | BbhTrackFive155CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 184 |
| × | BbhTrackFive156CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("")
Context: BbhTrackFive156CN: expected , got [(E)] |
Failed | IN1 103 OUT1 686 |
| × | BbhTrackFive157CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("")
Context: BbhTrackFive157CN: expected , got [(A)] |
Failed | IN1 115 OUT1 358 |
| ✓ | BbhTrackFive158CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 755 |
| ✓ | BbhTrackFive159CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 639 |
| ✓ | BbhTrackFive160CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 608 |
| ✓ | BbhTrackFive161CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 384 |
| ✓ | BbhTrackFive162CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 016 |
| ✓ | BbhTrackFive163CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 176 |
| ✓ | BbhTrackFive164CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 872 |
| ✓ | BbhTrackFive165CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 518 |
| ✓ | BbhTrackFive166CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT1 593 |
| ✓ | BbhTrackFive167CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 885 |
| ✓ | BbhTrackFive168CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 346 |
Reasoning - BBH tracking shuffled objects (5) (set 6)
32 passed |
100.0% |
IN35 543 OUT41 607 | |
| ✓ | BbhTrackFive169CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 263 |
| ✓ | BbhTrackFive170CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 831 |
| ✓ | BbhTrackFive171CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT1 498 |
| ✓ | BbhTrackFive172CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 784 |
| ✓ | BbhTrackFive173CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 296 |
| ✓ | BbhTrackFive174CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 093 |
| ✓ | BbhTrackFive175CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 117 |
| ✓ | BbhTrackFive176CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 377 |
| ✓ | BbhTrackFive177CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 117 |
| ✓ | BbhTrackFive178CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 233 |
| ✓ | BbhTrackFive179CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT1 415 |
| ✓ | BbhTrackFive180CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 348 |
| ✓ | BbhTrackFive181CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 315 |
| ✓ | BbhTrackFive182CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 096 |
| ✓ | BbhTrackFive183CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 270 |
| ✓ | BbhTrackFive184CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 002 |
| ✓ | BbhTrackFive185CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 160 |
| ✓ | BbhTrackFive186CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 410 |
| ✓ | BbhTrackFive187CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 152 |
| ✓ | BbhTrackFive188CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 234 |
| ✓ | BbhTrackFive189CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 205 |
| ✓ | BbhTrackFive190CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT994 |
| ✓ | BbhTrackFive191CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 963 |
| ✓ | BbhTrackFive192CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 118 |
| ✓ | BbhTrackFive193CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT985 |
| ✓ | BbhTrackFive194CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT972 |
| ✓ | BbhTrackFive195CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 432 |
| ✓ | BbhTrackFive196CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 896 |
| ✓ | BbhTrackFive197CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 198 |
| ✓ | BbhTrackFive198CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 161 |
| ✓ | BbhTrackFive199CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 534 |
| ✓ | BbhTrackFive200CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 138 |
Reasoning - BBH tracking shuffled objects (5) (set 7)
32 passed |
100.0% |
IN35 504 OUT46 483 | |
| ✓ | BbhTrackFive201CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 304 |
| ✓ | BbhTrackFive202CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 826 |
| ✓ | BbhTrackFive203CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 600 |
| ✓ | BbhTrackFive204CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 143 |
| ✓ | BbhTrackFive205CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 600 |
| ✓ | BbhTrackFive206CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 512 |
| ✓ | BbhTrackFive207CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 217 |
| ✓ | BbhTrackFive208CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 423 |
| ✓ | BbhTrackFive209CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 470 |
| ✓ | BbhTrackFive210CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 516 |
| ✓ | BbhTrackFive211CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT2 069 |
| ✓ | BbhTrackFive212CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 095 |
| ✓ | BbhTrackFive213CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 889 |
| ✓ | BbhTrackFive214CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 412 |
| ✓ | BbhTrackFive215CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 221 |
| ✓ | BbhTrackFive216CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT2 054 |
| ✓ | BbhTrackFive217CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 470 |
| ✓ | BbhTrackFive218CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 385 |
| ✓ | BbhTrackFive219CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 267 |
| ✓ | BbhTrackFive220CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 398 |
| ✓ | BbhTrackFive221CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 283 |
| ✓ | BbhTrackFive222CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 555 |
| ✓ | BbhTrackFive223CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 245 |
| ✓ | BbhTrackFive224CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 069 |
| ✓ | BbhTrackFive225CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 430 |
| ✓ | BbhTrackFive226CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 275 |
| ✓ | BbhTrackFive227CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 248 |
| ✓ | BbhTrackFive228CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 211 |
| ✓ | BbhTrackFive229CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 209 |
| ✓ | BbhTrackFive230CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT2 175 |
| ✓ | BbhTrackFive231CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 709 |
| ✓ | BbhTrackFive232CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 203 |
Reasoning - BBH tracking shuffled objects (5) (set 8)
32 passed |
100.0% |
IN36 208 OUT55 685 | |
| ✓ | BbhTrackFive233CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 517 |
| ✓ | BbhTrackFive234CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT1 050 |
| ✓ | BbhTrackFive235CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 232 |
| ✓ | BbhTrackFive236CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 257 |
| ✓ | BbhTrackFive237CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 230 |
| ✓ | BbhTrackFive238CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 321 |
| ✓ | BbhTrackFive239CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 287 |
| ✓ | BbhTrackFive240CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 636 |
| ✓ | BbhTrackFive241CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 873 |
| ✓ | BbhTrackFive242CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 739 |
| ✓ | BbhTrackFive243CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 601 |
| ✓ | BbhTrackFive244CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT1 302 |
| ✓ | BbhTrackFive245CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 062 |
| ✓ | BbhTrackFive246CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 144 |
| ✓ | BbhTrackFive247CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 682 |
| ✓ | BbhTrackFive248CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 992 |
| ✓ | BbhTrackFive249CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 368 |
| ✓ | BbhTrackSeven10CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT2 700 |
| ✓ | BbhTrackSeven11CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT2 854 |
| ✓ | BbhTrackSeven12CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 571 |
| ✓ | BbhTrackSeven13CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 675 |
| ✓ | BbhTrackSeven14CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 178 OUT1 897 |
| ✓ | BbhTrackSeven15CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 047 |
| ✓ | BbhTrackSeven16CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 817 |
| ✓ | BbhTrackSeven17CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 967 |
| ✓ | BbhTrackSeven18CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 341 |
| ✓ | BbhTrackSeven19CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 725 |
| ✓ | BbhTrackSeven20CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT2 010 |
| ✓ | BbhTrackSeven21CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 906 |
| ✓ | BbhTrackSeven22CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT2 295 |
| ✓ | BbhTrackSeven8CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 052 |
| ✓ | BbhTrackSeven9CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 535 |
Reasoning - BBH tracking shuffled objects (7) (set 1)
30 passed |
100.0% |
IN34 736 OUT54 136 | |
| ✓ | BbhTrackSeven23CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 691 |
| ✓ | BbhTrackSeven24CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 933 |
| ✓ | BbhTrackSeven25CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT2 212 |
| ✓ | BbhTrackSeven26CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 447 |
| ✓ | BbhTrackSeven27CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT2 202 |
| ✓ | BbhTrackSeven28CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT3 000 |
| ✓ | BbhTrackSeven29CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 086 |
| ✓ | BbhTrackSeven30CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 469 |
| ✓ | BbhTrackSeven31CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 581 |
| ✓ | BbhTrackSeven32CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 462 |
| ✓ | BbhTrackSeven33CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 272 |
| ✓ | BbhTrackSeven34CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 865 |
| ✓ | BbhTrackSeven35CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 931 |
| ✓ | BbhTrackSeven36CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 598 |
| ✓ | BbhTrackSeven37CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 132 |
| ✓ | BbhTrackSeven38CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 870 |
| ✓ | BbhTrackSeven39CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 717 |
| ✓ | BbhTrackSeven40CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 949 |
| ✓ | BbhTrackSeven41CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 212 |
| ✓ | BbhTrackSeven42CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 073 |
| ✓ | BbhTrackSeven43CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 660 |
| ✓ | BbhTrackSeven44CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 908 |
| ✓ | BbhTrackSeven45CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 773 |
| ✓ | BbhTrackSeven46CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 095 |
| ✓ | BbhTrackSeven47CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT1 627 |
| ✓ | BbhTrackSeven48CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 803 |
| ✓ | BbhTrackSeven49CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 231 |
| ✓ | BbhTrackSeven50CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 558 |
| ✓ | BbhTrackSeven51CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT1 303 |
| ✓ | BbhTrackSeven52CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 476 |
Reasoning - BBH tracking shuffled objects (7) (set 2)
32 passed |
100.0% |
IN37 082 OUT55 785 | |
| ✓ | BbhTrackingSeven53COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 257 |
| ✓ | BbhTrackingSeven54COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 176 OUT1 611 |
| ✓ | BbhTrackingSeven55COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 493 |
| ✓ | BbhTrackingSeven56COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 611 |
| ✓ | BbhTrackingSeven57COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT2 047 |
| ✓ | BbhTrackingSeven58COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 682 |
| ✓ | BbhTrackingSeven59COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT1 669 |
| ✓ | BbhTrackingSeven60COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 226 |
| ✓ | BbhTrackingSeven61COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 446 |
| ✓ | BbhTrackingSeven62COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 643 |
| ✓ | BbhTrackingSeven63COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 708 |
| ✓ | BbhTrackingSeven64COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 997 |
| ✓ | BbhTrackingSeven65COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 232 |
| ✓ | BbhTrackingSeven66COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 792 |
| ✓ | BbhTrackingSeven67COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 757 |
| ✓ | BbhTrackingSeven68COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 063 |
| ✓ | BbhTrackingSeven69COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 140 |
| ✓ | BbhTrackingSeven70COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 108 |
| ✓ | BbhTrackingSeven71COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 603 |
| ✓ | BbhTrackingSeven72COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT2 096 |
| ✓ | BbhTrackingSeven73COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 168 OUT1 543 |
| ✓ | BbhTrackingSeven74COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 553 |
| ✓ | BbhTrackingSeven75COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 789 |
| ✓ | BbhTrackingSeven76COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT2 083 |
| ✓ | BbhTrackingSeven77COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT2 064 |
| ✓ | BbhTrackingSeven78COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 932 |
| ✓ | BbhTrackingSeven79COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 741 |
| ✓ | BbhTrackingSeven80COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 719 |
| ✓ | BbhTrackingSeven81COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 435 |
| ✓ | BbhTrackingSeven82COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 594 |
| ✓ | BbhTrackingSeven83COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 662 |
| ✓ | BbhTrackingSeven84COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 489 |
Reasoning - BBH tracking shuffled objects (7) (set 3)
32 passed |
100.0% |
IN37 169 OUT61 399 | |
| ✓ | BbhTrackingSeven100COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT2 079 |
| ✓ | BbhTrackingSeven101COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 184 OUT1 982 |
| ✓ | BbhTrackingSeven102COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 405 |
| ✓ | BbhTrackingSeven103COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT1 501 |
| ✓ | BbhTrackingSeven104COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 465 |
| ✓ | BbhTrackingSeven105COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 844 |
| ✓ | BbhTrackingSeven106COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT3 817 |
| ✓ | BbhTrackingSeven107COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 161 |
| ✓ | BbhTrackingSeven108COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 722 |
| ✓ | BbhTrackingSeven109COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT1 826 |
| ✓ | BbhTrackingSeven110COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 699 |
| ✓ | BbhTrackingSeven111COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 360 |
| ✓ | BbhTrackingSeven112COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 434 |
| ✓ | BbhTrackingSeven113COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 007 |
| ✓ | BbhTrackingSeven114COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 662 |
| ✓ | BbhTrackingSeven115COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 178 OUT2 193 |
| ✓ | BbhTrackingSeven116COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 701 |
| ✓ | BbhTrackingSeven85COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 837 |
| ✓ | BbhTrackingSeven86COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT1 282 |
| ✓ | BbhTrackingSeven87COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 159 |
| ✓ | BbhTrackingSeven88COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 016 |
| ✓ | BbhTrackingSeven89COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 141 |
| ✓ | BbhTrackingSeven90COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 902 |
| ✓ | BbhTrackingSeven91COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT2 333 |
| ✓ | BbhTrackingSeven92COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 634 |
| ✓ | BbhTrackingSeven93COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 648 |
| ✓ | BbhTrackingSeven94COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 583 |
| ✓ | BbhTrackingSeven95COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 538 |
| ✓ | BbhTrackingSeven96COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 502 |
| ✓ | BbhTrackingSeven97COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT4 187 |
| ✓ | BbhTrackingSeven98COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 131 |
| ✓ | BbhTrackingSeven99COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 648 |
Reasoning - BBH tracking shuffled objects (7) (set 4)
32 passed |
100.0% |
IN37 165 OUT60 101 | |
| ✓ | BbhTrackingSeven117COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT2 096 |
| ✓ | BbhTrackingSeven118COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 485 |
| ✓ | BbhTrackingSeven119COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 578 |
| ✓ | BbhTrackingSeven120COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 444 |
| ✓ | BbhTrackingSeven121COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 021 |
| ✓ | BbhTrackingSeven122COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 295 |
| ✓ | BbhTrackingSeven123COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 417 |
| ✓ | BbhTrackingSeven124COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 168 OUT2 179 |
| ✓ | BbhTrackingSeven125COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 047 |
| ✓ | BbhTrackingSeven126COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 939 |
| ✓ | BbhTrackingSeven127COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 143 |
| ✓ | BbhTrackingSeven128COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 178 OUT2 403 |
| ✓ | BbhTrackingSeven129COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 154 OUT1 700 |
| ✓ | BbhTrackingSeven130COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT1 528 |
| ✓ | BbhTrackingSeven131COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT1 773 |
| ✓ | BbhTrackingSeven132COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 518 |
| ✓ | BbhTrackingSeven133COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 390 |
| ✓ | BbhTrackingSeven134COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 624 |
| ✓ | BbhTrackingSeven135COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 572 |
| ✓ | BbhTrackingSeven136COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 074 |
| ✓ | BbhTrackingSeven137COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT2 390 |
| ✓ | BbhTrackingSeven138COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT3 039 |
| ✓ | BbhTrackingSeven139COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 182 OUT1 389 |
| ✓ | BbhTrackingSeven140COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT1 610 |
| ✓ | BbhTrackingSeven141COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 141 |
| ✓ | BbhTrackingSeven142COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT1 952 |
| ✓ | BbhTrackingSeven143COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 518 |
| ✓ | BbhTrackingSeven144COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 170 OUT2 427 |
| ✓ | BbhTrackingSeven145COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 339 |
| ✓ | BbhTrackingSeven146COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 324 |
| ✓ | BbhTrackingSeven147COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 102 |
| ✓ | BbhTrackingSeven148COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 644 |
Reasoning - BBH tracking shuffled objects (7) (set 5)
31 passed |
100.0% |
IN35 985 OUT56 733 | |
| ✓ | BbhTrackingSeven149COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 281 |
| ✓ | BbhTrackingSeven150COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 947 |
| ✓ | BbhTrackingSeven151COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 667 |
| ✓ | BbhTrackingSeven152COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 010 |
| ✓ | BbhTrackingSeven153COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 059 |
| ✓ | BbhTrackingSeven154COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT1 861 |
| ✓ | BbhTrackingSeven155COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT2 060 |
| ✓ | BbhTrackingSeven156COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT2 117 |
| ✓ | BbhTrackingSeven157COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT2 480 |
| ✓ | BbhTrackingSeven158COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 893 |
| ✓ | BbhTrackingSeven159COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 954 |
| ✓ | BbhTrackingSeven160COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 767 |
| ✓ | BbhTrackingSeven161COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 927 |
| ✓ | BbhTrackingSeven162COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 945 |
| ✓ | BbhTrackingSeven163COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 535 |
| ✓ | BbhTrackingSeven164COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 591 |
| ✓ | BbhTrackingSeven165COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 619 |
| ✓ | BbhTrackingSeven166COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 484 |
| ✓ | BbhTrackingSeven167COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 597 |
| ✓ | BbhTrackingSeven168COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 835 |
| ✓ | BbhTrackingSeven169COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 624 |
| ✓ | BbhTrackingSeven170COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 491 |
| ✓ | BbhTrackingSeven171COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 519 |
| ✓ | BbhTrackingSeven172COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT1 830 |
| ✓ | BbhTrackingSeven173COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 703 |
| ✓ | BbhTrackingSeven174COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 708 |
| ✓ | BbhTrackingSeven175COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 505 |
| ✓ | BbhTrackingSeven176COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 168 OUT2 364 |
| ✓ | BbhTrackingSeven177COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 747 |
| ✓ | BbhTrackingSeven178COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 820 |
| ✓ | BbhTrackingSeven179COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 793 |
Reasoning - BBH tracking shuffled objects (7) (set 6)
31 passed |
100.0% |
IN35 993 OUT59 699 | |
| ✓ | BbhTrackingSeven180COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 199 |
| ✓ | BbhTrackingSeven181COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 392 |
| ✓ | BbhTrackingSeven182COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 429 |
| ✓ | BbhTrackingSeven183COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 140 |
| ✓ | BbhTrackingSeven184COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 356 |
| ✓ | BbhTrackingSeven185COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 668 |
| ✓ | BbhTrackingSeven186COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 008 |
| ✓ | BbhTrackingSeven187COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 764 |
| ✓ | BbhTrackingSeven188COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 942 |
| ✓ | BbhTrackingSeven189COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT2 071 |
| ✓ | BbhTrackingSeven190COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 879 |
| ✓ | BbhTrackingSeven191COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 596 |
| ✓ | BbhTrackingSeven192COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 158 |
| ✓ | BbhTrackingSeven193COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 126 |
| ✓ | BbhTrackingSeven194COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 723 |
| ✓ | BbhTrackingSeven195COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 612 |
| ✓ | BbhTrackingSeven196COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 924 |
| ✓ | BbhTrackingSeven197COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 656 |
| ✓ | BbhTrackingSeven198COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 427 |
| ✓ | BbhTrackingSeven199COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 259 |
| ✓ | BbhTrackingSeven200COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 946 |
| ✓ | BbhTrackingSeven201COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT2 562 |
| ✓ | BbhTrackingSeven202COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 043 |
| ✓ | BbhTrackingSeven203COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 460 |
| ✓ | BbhTrackingSeven204COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 857 |
| ✓ | BbhTrackingSeven205COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 131 |
| ✓ | BbhTrackingSeven206COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 281 |
| ✓ | BbhTrackingSeven207COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 335 |
| ✓ | BbhTrackingSeven208COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 915 |
| ✓ | BbhTrackingSeven209COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT3 035 |
| ✓ | BbhTrackingSeven210COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 805 |
Reasoning - BBH tracking shuffled objects (7) (set 7)
39 passed |
100.0% |
IN45 287 OUT72 902 | |
| ✓ | BbhTrackingSeven211CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 321 |
| ✓ | BbhTrackingSeven212CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 847 |
| ✓ | BbhTrackingSeven213CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 968 |
| ✓ | BbhTrackingSeven214CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 021 |
| ✓ | BbhTrackingSeven215CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 545 |
| ✓ | BbhTrackingSeven216CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 815 |
| ✓ | BbhTrackingSeven217CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 168 OUT1 523 |
| ✓ | BbhTrackingSeven218CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 541 |
| ✓ | BbhTrackingSeven219CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT2 287 |
| ✓ | BbhTrackingSeven220CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT2 159 |
| ✓ | BbhTrackingSeven221CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT2 232 |
| ✓ | BbhTrackingSeven222CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 402 |
| ✓ | BbhTrackingSeven223CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT2 441 |
| ✓ | BbhTrackingSeven224CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 349 |
| ✓ | BbhTrackingSeven225CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 170 OUT1 524 |
| ✓ | BbhTrackingSeven226CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 707 |
| ✓ | BbhTrackingSeven227CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 561 |
| ✓ | BbhTrackingSeven228CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT2 713 |
| ✓ | BbhTrackingSeven229CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 923 |
| ✓ | BbhTrackingSeven230CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 263 |
| ✓ | BbhTrackingSeven231CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT2 310 |
| ✓ | BbhTrackingSeven232CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 519 |
| ✓ | BbhTrackingSeven233CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 296 |
| ✓ | BbhTrackingSeven234CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 719 |
| ✓ | BbhTrackingSeven235CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 530 |
| ✓ | BbhTrackingSeven236CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 822 |
| ✓ | BbhTrackingSeven237CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 710 |
| ✓ | BbhTrackingSeven238CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 697 |
| ✓ | BbhTrackingSeven239CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 280 |
| ✓ | BbhTrackingSeven240CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 721 |
| ✓ | BbhTrackingSeven241CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT1 942 |
| ✓ | BbhTrackingSeven242CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT2 440 |
| ✓ | BbhTrackingSeven243CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 532 |
| ✓ | BbhTrackingSeven244CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 230 |
| ✓ | BbhTrackingSeven245CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 207 |
| ✓ | BbhTrackingSeven246CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 002 |
| ✓ | BbhTrackingSeven247CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT1 779 |
| ✓ | BbhTrackingSeven248CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 563 |
| ✓ | BbhTrackingSeven249CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 461 |
Reasoning - BBH web of lies (set 1)
34 passed1 failed |
97.1% |
IN34 922 OUT39 395 | |
| ✓ | BbhWebOfLies10CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 149 |
| ✓ | BbhWebOfLies11CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 012 |
| ✓ | BbhWebOfLies12CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 586 |
| ✓ | BbhWebOfLies13CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT728 |
| ✓ | BbhWebOfLies14CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT979 |
| ✓ | BbhWebOfLies15CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 255 |
| ✓ | BbhWebOfLies16CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 072 |
| ✓ | BbhWebOfLies17CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 274 |
| ✓ | BbhWebOfLies18CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT900 |
| ✓ | BbhWebOfLies19CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 034 |
| ✓ | BbhWebOfLies20CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 084 |
| ✓ | BbhWebOfLies21CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT866 |
| ✓ | BbhWebOfLies22CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 944 |
| ✓ | BbhWebOfLies23CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT947 |
| ✓ | BbhWebOfLies24CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT905 |
| ✓ | BbhWebOfLies25CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 122 |
| ✓ | BbhWebOfLies26CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 024 |
| ✓ | BbhWebOfLies27CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 652 |
| ✓ | BbhWebOfLies28CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT696 |
| × | BbhWebOfLies29CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("Yes")
Context: BbhWebOfLies29CQ: expected Yes, got [true]
Values:
got = "true" |
Failed | IN996 OUT1 005 |
| ✓ | BbhWebOfLies30CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT742 |
| ✓ | BbhWebOfLies31CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT735 |
| ✓ | BbhWebOfLies32CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 774 |
| ✓ | BbhWebOfLies33CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 044 |
| ✓ | BbhWebOfLies34CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 455 |
| ✓ | BbhWebOfLies35CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 017 |
| ✓ | BbhWebOfLies36CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT850 |
| ✓ | BbhWebOfLies37CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT711 |
| ✓ | BbhWebOfLies38CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 042 |
| ✓ | BbhWebOfLies39CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 173 |
| ✓ | BbhWebOfLies40CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 106 |
| ✓ | BbhWebOfLies41CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 267 |
| ✓ | BbhWebOfLies42CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 217 |
| ✓ | BbhWebOfLies8CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 228 |
| ✓ | BbhWebOfLies9CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT800 |
Reasoning - BBH web of lies (set 2)
32 passed |
100.0% |
IN31 928 OUT35 770 | |
| ✓ | BbhWebOfLies43CRtruth-teller/liar chain — Does Audrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT769 |
| ✓ | BbhWebOfLies44CRtruth-teller/liar chain — Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 243 |
| ✓ | BbhWebOfLies45CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT886 |
| ✓ | BbhWebOfLies46CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT2 073 |
| ✓ | BbhWebOfLies47CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 230 |
| ✓ | BbhWebOfLies48CRtruth-teller/liar chain — Does Maybelle tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 154 |
| ✓ | BbhWebOfLies49CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT898 |
| ✓ | BbhWebOfLies50CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT813 |
| ✓ | BbhWebOfLies51CRtruth-teller/liar chain — Does Sherrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT852 |
| ✓ | BbhWebOfLies52CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT994 |
| ✓ | BbhWebOfLies53CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT680 |
| ✓ | BbhWebOfLies54CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 091 |
| ✓ | BbhWebOfLies55CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT781 |
| ✓ | BbhWebOfLies56CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT618 |
| ✓ | BbhWebOfLies57CRtruth-teller/liar chain — Does Shalonda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT973 |
| ✓ | BbhWebOfLies58CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 385 |
| ✓ | BbhWebOfLies59CRtruth-teller/liar chain — Does Crista tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 322 |
| ✓ | BbhWebOfLies60CRtruth-teller/liar chain — Does Sima tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 131 |
| ✓ | BbhWebOfLies61CRtruth-teller/liar chain — Does Helene tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 216 |
| ✓ | BbhWebOfLies62CRtruth-teller/liar chain — Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT2 167 |
| ✓ | BbhWebOfLies63CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT832 |
| ✓ | BbhWebOfLies64CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 295 |
| ✓ | BbhWebOfLies65CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 260 |
| ✓ | BbhWebOfLies66CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT784 |
| ✓ | BbhWebOfLies67CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 316 |
| ✓ | BbhWebOfLies68CRtruth-teller/liar chain — Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT731 |
| ✓ | BbhWebOfLies69CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 194 |
| ✓ | BbhWebOfLies70CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT982 |
| ✓ | BbhWebOfLies71CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 447 |
| ✓ | BbhWebOfLies72CRtruth-teller/liar chain — Does Vernell tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 250 |
| ✓ | BbhWebOfLies73CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 395 |
| ✓ | BbhWebOfLies74CRtruth-teller/liar chain — Does Jaymie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 008 |
Reasoning - BBH web of lies (set 3)
32 passed |
100.0% |
IN31 932 OUT36 867 | |
| ✓ | BbhWebOfLies100CRtruth-teller/liar chain — Does Fidel tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 144 |
| ✓ | BbhWebOfLies101CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 129 |
| ✓ | BbhWebOfLies102CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 601 |
| ✓ | BbhWebOfLies103CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 800 |
| ✓ | BbhWebOfLies104CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 313 |
| ✓ | BbhWebOfLies105CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 574 |
| ✓ | BbhWebOfLies106CRtruth-teller/liar chain — Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 269 |
| ✓ | BbhWebOfLies75CRtruth-teller/liar chain — Does Crista tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 225 |
| ✓ | BbhWebOfLies76CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT819 |
| ✓ | BbhWebOfLies77CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT832 |
| ✓ | BbhWebOfLies78CRtruth-teller/liar chain — Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT980 |
| ✓ | BbhWebOfLies79CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT730 |
| ✓ | BbhWebOfLies80CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 318 |
| ✓ | BbhWebOfLies81CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 011 |
| ✓ | BbhWebOfLies82CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 018 |
| ✓ | BbhWebOfLies83CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 169 |
| ✓ | BbhWebOfLies84CRtruth-teller/liar chain — Does Michael tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 605 |
| ✓ | BbhWebOfLies85CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT825 |
| ✓ | BbhWebOfLies86CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT939 |
| ✓ | BbhWebOfLies87CRtruth-teller/liar chain — Does Leda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 587 |
| ✓ | BbhWebOfLies88CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 003 |
| ✓ | BbhWebOfLies89CRtruth-teller/liar chain — Does Fletcher tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT870 |
| ✓ | BbhWebOfLies90CRtruth-teller/liar chain — Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 582 |
| ✓ | BbhWebOfLies91CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 069 |
| ✓ | BbhWebOfLies92CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 013 |
| ✓ | BbhWebOfLies93CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 189 |
| ✓ | BbhWebOfLies94CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT943 |
| ✓ | BbhWebOfLies95CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT672 |
| ✓ | BbhWebOfLies96CRtruth-teller/liar chain — Does Phoebe tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT880 |
| ✓ | BbhWebOfLies97CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 798 |
| ✓ | BbhWebOfLies98CRtruth-teller/liar chain — Does Leda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT747 |
| ✓ | BbhWebOfLies99CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 213 |
Reasoning - BBH web of lies (set 4)
32 passed |
100.0% |
IN31 952 OUT38 166 | |
| ✓ | BbhWebOfLies107CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT833 |
| ✓ | BbhWebOfLies108CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 949 |
| ✓ | BbhWebOfLies109CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 066 |
| ✓ | BbhWebOfLies110CRtruth-teller/liar chain — Does Kristian tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 759 |
| ✓ | BbhWebOfLies111CRtruth-teller/liar chain — Does Crista tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT795 |
| ✓ | BbhWebOfLies112CRtruth-teller/liar chain — Does Bernita tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 013 |
| ✓ | BbhWebOfLies113CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 345 |
| ✓ | BbhWebOfLies114CRtruth-teller/liar chain — Does Millie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT874 |
| ✓ | BbhWebOfLies115CRtruth-teller/liar chain — Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT820 |
| ✓ | BbhWebOfLies116CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 399 |
| ✓ | BbhWebOfLies117CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT965 |
| ✓ | BbhWebOfLies118CRtruth-teller/liar chain — Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT820 |
| ✓ | BbhWebOfLies119CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT699 |
| ✓ | BbhWebOfLies120CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT827 |
| ✓ | BbhWebOfLies121CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 389 |
| ✓ | BbhWebOfLies122CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 406 |
| ✓ | BbhWebOfLies123CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 234 |
| ✓ | BbhWebOfLies124CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 339 |
| ✓ | BbhWebOfLies125CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT802 |
| ✓ | BbhWebOfLies126CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 722 |
| ✓ | BbhWebOfLies127CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 356 |
| ✓ | BbhWebOfLies128CRtruth-teller/liar chain — Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 956 |
| ✓ | BbhWebOfLies129CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 160 |
| ✓ | BbhWebOfLies130CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT801 |
| ✓ | BbhWebOfLies131CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 571 |
| ✓ | BbhWebOfLies132CRtruth-teller/liar chain — Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT834 |
| ✓ | BbhWebOfLies133CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT812 |
| ✓ | BbhWebOfLies134CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 597 |
| ✓ | BbhWebOfLies135CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 015 |
| ✓ | BbhWebOfLies136CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT981 |
| ✓ | BbhWebOfLies137CRtruth-teller/liar chain — Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 427 |
| ✓ | BbhWebOfLies138CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 600 |
Reasoning - BBH web of lies (set 5)
31 passed1 failed |
96.9% |
IN31 946 OUT34 758 | |
| ✓ | BbhWebOfLies139CRtruth-teller/liar chain — Does Fletcher tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT928 |
| ✓ | BbhWebOfLies140CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 007 |
| ✓ | BbhWebOfLies141CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 193 |
| ✓ | BbhWebOfLies142CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 212 |
| ✓ | BbhWebOfLies143CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT916 |
| ✓ | BbhWebOfLies144CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT866 |
| ✓ | BbhWebOfLies145CRtruth-teller/liar chain — Does Fidel tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT754 |
| × | BbhWebOfLies146CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("Yes")
Context: BbhWebOfLies146CR: expected Yes, got [true]
Values:
got = "true" |
Failed | IN996 OUT1 033 |
| ✓ | BbhWebOfLies147CRtruth-teller/liar chain — Does Millie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 264 |
| ✓ | BbhWebOfLies148CRtruth-teller/liar chain — Does Audrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 146 |
| ✓ | BbhWebOfLies149CRtruth-teller/liar chain — Does Phoebe tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT984 |
| ✓ | BbhWebOfLies150CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 427 |
| ✓ | BbhWebOfLies151CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 326 |
| ✓ | BbhWebOfLies152CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 268 |
| ✓ | BbhWebOfLies153CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT747 |
| ✓ | BbhWebOfLies154CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT913 |
| ✓ | BbhWebOfLies155CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT689 |
| ✓ | BbhWebOfLies156CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 025 |
| ✓ | BbhWebOfLies157CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT419 |
| ✓ | BbhWebOfLies158CRtruth-teller/liar chain — Does Shalonda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 192 |
| ✓ | BbhWebOfLies159CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT2 833 |
| ✓ | BbhWebOfLies160CRtruth-teller/liar chain — Does Ryan tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 016 |
| ✓ | BbhWebOfLies161CRtruth-teller/liar chain — Does Willian tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 134 |
| ✓ | BbhWebOfLies162CRtruth-teller/liar chain — Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT976 |
| ✓ | BbhWebOfLies163CRtruth-teller/liar chain — Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 121 |
| ✓ | BbhWebOfLies164CRtruth-teller/liar chain — Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT879 |
| ✓ | BbhWebOfLies165CRtruth-teller/liar chain — Does Raymond tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 111 |
| ✓ | BbhWebOfLies166CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT868 |
| ✓ | BbhWebOfLies167CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT723 |
| ✓ | BbhWebOfLies168CRtruth-teller/liar chain — Does Leda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 619 |
| ✓ | BbhWebOfLies169CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 109 |
| ✓ | BbhWebOfLies170CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 060 |
Reasoning - BBH web of lies (set 6)
29 passed1 failed |
96.7% |
IN29 926 OUT37 010 | |
| ✓ | BbhWebOfLies171CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT935 |
| ✓ | BbhWebOfLies172CRtruth-teller/liar chain — Does Bernita tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 124 |
| ✓ | BbhWebOfLies173CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 459 |
| ✓ | BbhWebOfLies174CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 546 |
| ✓ | BbhWebOfLies175CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 007 |
| ✓ | BbhWebOfLies176CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT870 |
| ✓ | BbhWebOfLies177CRtruth-teller/liar chain — Does Phoebe tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 968 |
| ✓ | BbhWebOfLies178CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 163 |
| ✓ | BbhWebOfLies179CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 092 |
| × | BbhWebOfLies180CRtruth-teller/liar chain — Does Sherrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("Yes")
Context: BbhWebOfLies180CR: expected Yes, got [true]
Values:
got = "true" |
Failed | IN1 004 OUT1 083 |
| ✓ | BbhWebOfLies181CRtruth-teller/liar chain — Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 147 |
| ✓ | BbhWebOfLies182CRtruth-teller/liar chain — Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT975 |
| ✓ | BbhWebOfLies183CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 019 |
| ✓ | BbhWebOfLies184CRtruth-teller/liar chain — Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 104 |
| ✓ | BbhWebOfLies185CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 173 |
| ✓ | BbhWebOfLies186CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT939 |
| ✓ | BbhWebOfLies187CRtruth-teller/liar chain — Does Fletcher tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT2 141 |
| ✓ | BbhWebOfLies188CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 129 |
| ✓ | BbhWebOfLies189CRtruth-teller/liar chain — Does Helene tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 459 |
| ✓ | BbhWebOfLies190CRtruth-teller/liar chain — Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 776 |
| ✓ | BbhWebOfLies191CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT773 |
| ✓ | BbhWebOfLies192CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 059 |
| ✓ | BbhWebOfLies193CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 007 |
| ✓ | BbhWebOfLies194CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 603 |
| ✓ | BbhWebOfLies195CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 129 |
| ✓ | BbhWebOfLies196CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT821 |
| ✓ | BbhWebOfLies197CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT982 |
| ✓ | BbhWebOfLies198CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 403 |
| ✓ | BbhWebOfLies199CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 411 |
| ✓ | BbhWebOfLies200CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 713 |
Reasoning - BBH web of lies (set 7)
32 passed |
100.0% |
IN31 938 OUT39 127 | |
| ✓ | BbhWebOfLies201CSJamey tells the truth. Michael says Jamey lies. Sherrie says Michael lies. Bernita says Sherrie lies. Amberly says Bernita tells the truth. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 207 |
| ✓ | BbhWebOfLies202CSShaunda tells the truth. Jim says Shaunda lies. Alexis says Jim lies. Phoebe says Alexis tells the truth. Sherrie says Phoebe tells the truth. Does Sherrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 692 |
| ✓ | BbhWebOfLies203CSMillicent lies. Ka says Millicent lies. Sima says Ka tells the truth. Dallas says Sima tells the truth. Jamey says Dallas lies. Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT990 |
| ✓ | BbhWebOfLies204CSChristie tells the truth. Fidel says Christie lies. Jerry says Fidel tells the truth. Maybelle says Jerry lies. Delfina says Maybelle tells the truth. Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 046 |
| ✓ | BbhWebOfLies205CSSima lies. Maybelle says Sima tells the truth. Audrie says Maybelle lies. Leda says Audrie tells the truth. Shaunda says Leda lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 072 |
| ✓ | BbhWebOfLies206CSVernell lies. Conception says Vernell lies. Kandi says Conception lies. Michael says Kandi tells the truth. Osvaldo says Michael tells the truth. Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 026 |
| ✓ | BbhWebOfLies207CSShaunda tells the truth. Delbert says Shaunda tells the truth. Bernita says Delbert lies. Yoland says Bernita tells the truth. Christie says Yoland tells the truth. Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 430 |
| ✓ | BbhWebOfLies208CSInga tells the truth. Shalonda says Inga tells the truth. Phoebe says Shalonda tells the truth. Crista says Phoebe lies. Alejandro says Crista lies. Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT929 |
| ✓ | BbhWebOfLies209CSFidel lies. Millicent says Fidel tells the truth. Crista says Millicent lies. Leda says Crista lies. Tamika says Leda tells the truth. Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 081 |
| ✓ | BbhWebOfLies210CSAndree tells the truth. Christie says Andree tells the truth. Fidel says Christie lies. Antwan says Fidel tells the truth. Alejandro says Antwan lies. Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 240 |
| ✓ | BbhWebOfLies211CSFletcher tells the truth. Willian says Fletcher tells the truth. Jamey says Willian lies. Leda says Jamey lies. Jaymie says Leda tells the truth. Does Jaymie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT792 |
| ✓ | BbhWebOfLies212CSTamika lies. Jerry says Tamika tells the truth. Delbert says Jerry tells the truth. Leda says Delbert tells the truth. Willian says Leda lies. Does Willian tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT781 |
| ✓ | BbhWebOfLies213CSTeressa lies. Sal says Teressa tells the truth. Delbert says Sal tells the truth. Tamika says Delbert tells the truth. Gwenn says Tamika tells the truth. Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 524 |
| ✓ | BbhWebOfLies214CSJim lies. Delfina says Jim tells the truth. Shaunda says Delfina lies. Fidel says Shaunda lies. Sal says Fidel tells the truth. Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 734 |
| ✓ | BbhWebOfLies215CSMillie lies. Michael says Millie tells the truth. Vina says Michael lies. Conception says Vina tells the truth. Sima says Conception tells the truth. Does Sima tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 190 |
| ✓ | BbhWebOfLies216CSKandi lies. Millie says Kandi tells the truth. Lorine says Millie tells the truth. Michaela says Lorine tells the truth. Amberly says Michaela lies. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 361 |
| ✓ | BbhWebOfLies217CSMichael lies. Jerry says Michael tells the truth. Delbert says Jerry tells the truth. Sima says Delbert lies. Alexis says Sima lies. Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 587 |
| ✓ | BbhWebOfLies218CSSal lies. Kristian says Sal lies. Helene says Kristian lies. Inga says Helene tells the truth. Michael says Inga tells the truth. Does Michael tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT836 |
| ✓ | BbhWebOfLies219CSMaybelle lies. Crista says Maybelle tells the truth. Bernita says Crista lies. Kandi says Bernita tells the truth. Delbert says Kandi tells the truth. Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 153 |
| ✓ | BbhWebOfLies220CSKa lies. Rashida says Ka tells the truth. Sherrie says Rashida tells the truth. Tamika says Sherrie tells the truth. Elanor says Tamika tells the truth. Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 374 |
| ✓ | BbhWebOfLies221CSAntwan tells the truth. Audrie says Antwan tells the truth. Inga says Audrie lies. Shenna says Inga tells the truth. Christie says Shenna lies. Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT910 |
| ✓ | BbhWebOfLies222CSTeressa lies. Michael says Teressa tells the truth. Fidel says Michael tells the truth. Leda says Fidel lies. Lorine says Leda lies. Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 096 |
| ✓ | BbhWebOfLies223CSAntwan tells the truth. Elanor says Antwan tells the truth. Bernita says Elanor tells the truth. Osvaldo says Bernita lies. Christie says Osvaldo tells the truth. Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 376 |
| ✓ | BbhWebOfLies224CSInga tells the truth. Elanor says Inga lies. Millicent says Elanor tells the truth. Willian says Millicent tells the truth. Alejandro says Willian tells the truth. Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT846 |
| ✓ | BbhWebOfLies225CSSherrie lies. Elanor says Sherrie tells the truth. Michael says Elanor tells the truth. Sima says Michael tells the truth. Shaunda says Sima lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 420 |
| ✓ | BbhWebOfLies226CSTamika lies. Conception says Tamika tells the truth. Kristian says Conception lies. Alejandro says Kristian tells the truth. Shaunda says Alejandro lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT957 |
| ✓ | BbhWebOfLies227CSHelene lies. Delfina says Helene lies. Christie says Delfina tells the truth. Jerry says Christie lies. Tamika says Jerry tells the truth. Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 500 |
| ✓ | BbhWebOfLies228CSConception lies. Shenna says Conception tells the truth. Maybelle says Shenna tells the truth. Michael says Maybelle lies. Amberly says Michael lies. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 686 |
| ✓ | BbhWebOfLies229CSKristian tells the truth. Jamey says Kristian tells the truth. Millicent says Jamey lies. Fidel says Millicent lies. Tamika says Fidel lies. Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 012 |
| ✓ | BbhWebOfLies230CSJamey lies. Sal says Jamey tells the truth. Teressa says Sal lies. Maybelle says Teressa tells the truth. Ka says Maybelle lies. Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 580 |
| ✓ | BbhWebOfLies231CSLorine lies. Amberly says Lorine lies. Fletcher says Amberly lies. Millie says Fletcher tells the truth. Antwan says Millie lies. Does Antwan tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 639 |
| ✓ | BbhWebOfLies232CSAlexis tells the truth. Elanor says Alexis tells the truth. Sherrie says Elanor lies. Audrie says Sherrie tells the truth. Inga says Audrie tells the truth. Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 060 |
Reasoning - BBH web of lies (set 8)
32 passed |
100.0% |
IN31 665 OUT33 552 | |
| ✓ | BbhWebOfLies233CSFidel tells the truth. Teressa says Fidel tells the truth. Willian says Teressa tells the truth. Conception says Willian tells the truth. Bernita says Conception lies. Does Bernita tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 172 |
| ✓ | BbhWebOfLies234CSJerry tells the truth. Lorine says Jerry lies. Shenna says Lorine lies. Gwenn says Shenna lies. Shaunda says Gwenn lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT739 |
| ✓ | BbhWebOfLies235CSInga lies. Audrie says Inga lies. Rashida says Audrie lies. Willian says Rashida tells the truth. Amberly says Willian lies. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT955 |
| ✓ | BbhWebOfLies236CSKa tells the truth. Inga says Ka lies. Bernita says Inga lies. Osvaldo says Bernita tells the truth. Michaela says Osvaldo lies. Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 186 |
| ✓ | BbhWebOfLies237CSJaymie lies. Jamey says Jaymie tells the truth. Sal says Jamey lies. Amberly says Sal lies. Jerry says Amberly tells the truth. Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 263 |
| ✓ | BbhWebOfLies238CSMillie lies. Andree says Millie lies. Michaela says Andree lies. Leda says Michaela tells the truth. Shenna says Leda lies. Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 524 |
| ✓ | BbhWebOfLies239CSShaunda tells the truth. Maybelle says Shaunda tells the truth. Lorine says Maybelle lies. Millie says Lorine tells the truth. Vina says Millie tells the truth. Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 321 |
| ✓ | BbhWebOfLies240CSFletcher lies. Tamika says Fletcher tells the truth. Ka says Tamika lies. Sima says Ka tells the truth. Ryan says Sima tells the truth. Does Ryan tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT802 |
| ✓ | BbhWebOfLies241CSShaunda lies. Andree says Shaunda tells the truth. Jaymie says Andree lies. Inga says Jaymie lies. Millicent says Inga tells the truth. Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 381 |
| ✓ | BbhWebOfLies242CSRashida lies. Fletcher says Rashida lies. Antwan says Fletcher lies. Willian says Antwan tells the truth. Vina says Willian tells the truth. Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 349 |
| ✓ | BbhWebOfLies243CSJerry lies. Kandi says Jerry lies. Fletcher says Kandi lies. Ryan says Fletcher lies. Delfina says Ryan lies. Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 042 |
| ✓ | BbhWebOfLies244CSShaunda tells the truth. Phoebe says Shaunda tells the truth. Ryan says Phoebe tells the truth. Christie says Ryan tells the truth. Vina says Christie tells the truth. Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT768 |
| ✓ | BbhWebOfLies245CSAlexis tells the truth. Amberly says Alexis tells the truth. Andree says Amberly tells the truth. Vina says Andree tells the truth. Helene says Vina tells the truth. Does Helene tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 411 |
| ✓ | BbhWebOfLies246CSGwenn lies. Maybelle says Gwenn tells the truth. Raymond says Maybelle tells the truth. Jamey says Raymond tells the truth. Andree says Jamey lies. Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT826 |
| ✓ | BbhWebOfLies247CSKandi tells the truth. Ryan says Kandi lies. Michaela says Ryan tells the truth. Sima says Michaela lies. Andree says Sima tells the truth. Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 076 |
| ✓ | BbhWebOfLies248CSFletcher tells the truth. Alexis says Fletcher lies. Jerry says Alexis lies. Elanor says Jerry tells the truth. Jim says Elanor lies. Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT881 |
| ✓ | BbhWebOfLies249CSSherrie tells the truth. Ryan says Sherrie lies. Bernita says Ryan tells the truth. Tamika says Bernita lies. Jerry says Tamika lies. Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 098 |
| ✓ | BbhWordSorting10CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT702 |
| ✓ | BbhWordSorting11CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT1 417 |
| ✓ | BbhWordSorting12CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT940 |
| ✓ | BbhWordSorting13CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT826 |
| ✓ | BbhWordSorting14CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT1 066 |
| ✓ | BbhWordSorting15CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 415 |
| ✓ | BbhWordSorting16CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 001 OUT2 508 |
| ✓ | BbhWordSorting17CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT810 |
| ✓ | BbhWordSorting18CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT990 |
| ✓ | BbhWordSorting19CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT692 |
| ✓ | BbhWordSorting20CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT379 |
| ✓ | BbhWordSorting21CSalphabetically sort 7 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 044 |
| ✓ | BbhWordSorting22CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT899 |
| ✓ | BbhWordSorting8CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT358 |
| ✓ | BbhWordSorting9CSalphabetically sort 6 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT712 |
Reasoning - BBH word sorting (set 1)
32 passed |
100.0% |
IN31 387 OUT33 636 | |
| ✓ | BbhWordSorting23CSalphabetically sort 7 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 104 |
| ✓ | BbhWordSorting24CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 467 |
| ✓ | BbhWordSorting25CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT1 518 |
| ✓ | BbhWordSorting26CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT969 |
| ✓ | BbhWordSorting27CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT643 |
| ✓ | BbhWordSorting28CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT881 |
| ✓ | BbhWordSorting29CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 010 |
| ✓ | BbhWordSorting30CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT968 |
| ✓ | BbhWordSorting31CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT981 |
| ✓ | BbhWordSorting32CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN965 OUT292 |
| ✓ | BbhWordSorting33CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT2 268 |
| ✓ | BbhWordSorting34CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT570 |
| ✓ | BbhWordSorting35CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT2 015 |
| ✓ | BbhWordSorting36CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT941 |
| ✓ | BbhWordSorting37CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT2 735 |
| ✓ | BbhWordSorting38CSalphabetically sort 2 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT364 |
| ✓ | BbhWordSorting39CSalphabetically sort 2 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN964 OUT395 |
| ✓ | BbhWordSorting40CSalphabetically sort 14 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 139 |
| ✓ | BbhWordSorting41CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT941 |
| ✓ | BbhWordSorting42CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT398 |
| ✓ | BbhWordSorting43CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT812 |
| ✓ | BbhWordSorting44CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 872 |
| ✓ | BbhWordSorting45CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 418 |
| ✓ | BbhWordSorting46CSalphabetically sort 6 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT994 |
| ✓ | BbhWordSorting47CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 352 |
| ✓ | BbhWordSorting48CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT930 |
| ✓ | BbhWordSorting49CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 107 |
| ✓ | BbhWordSorting50CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT630 |
| ✓ | BbhWordSorting51CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT810 |
| ✓ | BbhWordSorting52CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT567 |
| ✓ | BbhWordSorting53CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT412 |
| ✓ | BbhWordSorting54CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 133 |
Reasoning - BBH word sorting (set 2)
28 passed4 failed |
87.5% |
IN31 603 OUT42 118 | |
| ✓ | BbhWordSorting55CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT1 181 |
| ✓ | BbhWordSorting56CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT712 |
| ✓ | BbhWordSorting57CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 010 OUT2 474 |
| ✓ | BbhWordSorting58CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 005 OUT2 062 |
| ✓ | BbhWordSorting59CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT1 787 |
| ✓ | BbhWordSorting60CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 273 |
| ✓ | BbhWordSorting61CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 559 |
| ✓ | BbhWordSorting62CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 001 OUT1 739 |
| ✓ | BbhWordSorting63CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT944 |
| ✓ | BbhWordSorting64CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT880 |
| ✓ | BbhWordSorting65CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 021 |
| ✓ | BbhWordSorting66CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 500 |
| ✓ | BbhWordSorting67CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 670 |
| ✓ | BbhWordSorting68CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT732 |
| × | BbhWordSorting69CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"avalanche befriend berniece bong bremsstrahlung dactylic flick gilbertson goff hereafter hoe housekeep hurry lanka metazoan posterior showroom")
Context: BbhWordSorting69CS: expected avalanche befriend berniece bong bremsstrahlung dactylic flick gilbertson goff hereafter hoe housekeep hurry lanka metazoan posterior showroom, got [avalanche befriend berniece bremsstrahlung bong dactylic flick gilbertson goff hereafter hoe housekeep hurry lanka metazoan posterior showroom] |
Failed | IN994 OUT1 362 |
| × | BbhWordSorting70CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"booby butadiene flair functor heck orphanage racy rheumatic shivery sin snowball spec testy trench zorn")
Context: BbhWordSorting70CS: expected booby butadiene flair functor heck orphanage racy rheumatic shivery sin snowball spec testy trench zorn, got [booby butadiene flair functor heck orphanage racy rheumatic sin shivery snowball spec testy trench zorn] |
Failed | IN987 OUT1 655 |
| ✓ | BbhWordSorting71CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 621 |
| ✓ | BbhWordSorting72CSalphabetically sort 8 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT1 291 |
| ✓ | BbhWordSorting73CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT1 506 |
| × | BbhWordSorting74CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"allot chauncey clergymen coachmen coddington companion embark fatten gazpacho granular hobble murk muslim niggle pristine pvc singlet threefold too yeats")
Context: BbhWordSorting74CS: expected allot chauncey clergymen coachmen coddington companion embark fatten gazpacho granular hobble murk muslim niggle pristine pvc singlet threefold too yeats, got [allot chauncey coachmen coddington clergymen companion embark fatten gazpacho granular hobble murk muslim niggle pristine pvc singlet threefold too yeats] |
Failed | IN1 001 OUT2 392 |
| ✓ | BbhWordSorting75CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT528 |
| ✓ | BbhWordSorting76CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT972 |
| ✓ | BbhWordSorting77CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT735 |
| ✓ | BbhWordSorting78CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 626 |
| ✓ | BbhWordSorting79CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 617 |
| ✓ | BbhWordSorting80CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 006 OUT1 328 |
| ✓ | BbhWordSorting81CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT1 165 |
| ✓ | BbhWordSorting82CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT550 |
| ✓ | BbhWordSorting83CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 108 |
| ✓ | BbhWordSorting84CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT637 |
| ✓ | BbhWordSorting85CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT891 |
| × | BbhWordSorting86CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"betony boar bootleg bronzy centaur charge clemens collet contemporaneous contravariant cordial dorado handicraft macmahon mesh monterey possession regina underclassman")
Context: BbhWordSorting86CS: expected betony boar bootleg bronzy centaur charge clemens collet contemporaneous contravariant cordial dorado handicraft macmahon mesh monterey possession regina underclassman, got [boar betony bootleg bronzy centaur charge clemens collet contemporaneous contravariant cordial dorado handicraft macmahon mesh monterey possession regina underclassman] |
Failed | IN1 000 OUT1 600 |
Reasoning - BBH word sorting (set 3)
24 passed6 failed |
80.0% |
IN29 591 OUT40 425 | |
| ✓ | BbhWordSorting100CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT424 |
| × | BbhWordSorting101CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"administer aeneid coachman decadent delhi dey gradate grim jacky littleneck phosphorescent pristine shrunk sinh systemwide tasting thrown torpedo verdict")
Context: BbhWordSorting101CS: expected administer aeneid coachman decadent delhi dey gradate grim jacky littleneck phosphorescent pristine shrunk sinh systemwide tasting thrown torpedo verdict, got [administer aeneid coachman decadent delhi dey gradate grim jacky littleneck phosphorescent pristine shrunk sinh systemwide tasting torpedo thrown verdict] |
Failed | IN993 OUT2 204 |
| × | BbhWordSorting102CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"adsorption align anastasia anastomotic apache award bobbin burrow calumny epaulet execrable hostelry hun macedon omnipotent putty roughshod smooth spontaneity")
Context: BbhWordSorting102CS: expected adsorption align anastasia anastomotic apache award bobbin burrow calumny epaulet execrable hostelry hun macedon omnipotent putty roughshod smooth spontaneity, got [adsorption align anastasia anastomotic apache award bobbin burrow calumny execrable epaulet hostelry hun macedon omnipotent putty roughshod smooth spontaneity] |
Failed | IN1 004 OUT1 560 |
| ✓ | BbhWordSorting103CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT929 |
| ✓ | BbhWordSorting104CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 203 |
| × | BbhWordSorting105CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"arenaceous baccarat blare bowman earl gloss granola hollandaise inauspicious mackenzie metaphoric pedro penis psyche quarantine roadster supranational")
Context: BbhWordSorting105CS: expected arenaceous baccarat blare bowman earl gloss granola hollandaise inauspicious mackenzie metaphoric pedro penis psyche quarantine roadster supranational, got [arenaceous baccarat blare bowman earl gloss granola hollandaise inauspicious mackenzie metaphoric penis pedro psyche quarantine roadster supranational] |
Failed | IN997 OUT1 892 |
| ✓ | BbhWordSorting106CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 662 |
| ✓ | BbhWordSorting107CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 412 |
| ✓ | BbhWordSorting108CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT1 316 |
| ✓ | BbhWordSorting109CSalphabetically sort 8 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT941 |
| ✓ | BbhWordSorting110CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 175 |
| ✓ | BbhWordSorting111CSalphabetically sort 14 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT1 373 |
| ✓ | BbhWordSorting112CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT1 826 |
| ✓ | BbhWordSorting113CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 882 |
| × | BbhWordSorting114CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"bijective briton concord dim dive eigenspace floruit gaucherie glycogen guidebook irrevocable jacket pinkish reversible song")
Context: BbhWordSorting114CS: expected bijective briton concord dim dive eigenspace floruit gaucherie glycogen guidebook irrevocable jacket pinkish reversible song, got [bijective briton concord dim dive eigenspace floruit gaucherie guidebook glycogen irrevocable jacket pinkish reversible song] |
Failed | IN990 OUT1 568 |
| ✓ | BbhWordSorting115CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT1 075 |
| ✓ | BbhWordSorting116CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 765 |
| ✓ | BbhWordSorting87CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 580 |
| ✓ | BbhWordSorting88CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT639 |
| ✓ | BbhWordSorting89CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT2 609 |
| × | BbhWordSorting90CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"assure bully butterball bye contend cornet deaf dinosaur frontage gunky indeterminable lustrous ostentatious paradigmatic rhyme sanderson sashimi smokestack taint")
Context: BbhWordSorting90CS: expected assure bully butterball bye contend cornet deaf dinosaur frontage gunky indeterminable lustrous ostentatious paradigmatic rhyme sanderson sashimi smokestack taint, got [assure bully butterball bye contend cornet deaf dinosaur frontage gunky indeterminable lustrous ostentatious paradigmatic rhyme sashimi sanderson smokestack taint] |
Failed | IN997 OUT1 300 |
| × | BbhWordSorting91CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"backpack coffman collision cotman detour gnostic hammock jacobean lung membrane monologist notate quirinal rhubarb secretive stove tobacco")
Context: BbhWordSorting91CS: expected backpack coffman collision cotman detour gnostic hammock jacobean lung membrane monologist notate quirinal rhubarb secretive stove tobacco, got [backpack collision coffman cotman detour gnostic hammock jacobean lung membrane monologist notate quirinal rhubarb secretive stove tobacco] |
Failed | IN991 OUT997 |
| ✓ | BbhWordSorting92CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT577 |
| ✓ | BbhWordSorting93CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 340 |
| ✓ | BbhWordSorting94CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 770 |
| ✓ | BbhWordSorting95CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 511 |
| ✓ | BbhWordSorting96CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 873 |
| ✓ | BbhWordSorting97CSalphabetically sort 14 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 414 |
| ✓ | BbhWordSorting98CSalphabetically sort 2 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN964 OUT205 |
| ✓ | BbhWordSorting99CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT403 |
Reasoning - BBH word sorting (set 4)
25 passed2 failed |
92.6% |
IN26 516 OUT28 441 | |
| ✓ | BbhWordSorting117CTSort the following words alphabetically: List: marshmallow doge alpenstock ocean accept angus drool jutish resistive chromium vociferous castigate prize octennial stonewort concision elizabethan1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT1 282 |
| ✓ | BbhWordSorting118CTSort the following words alphabetically: List: jocund flagellate bodyguard flotation commensal involve miff ineradicable postprocess1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT798 |
| ✓ | BbhWordSorting119CTSort the following words alphabetically: List: gloriana milord rendezvous schnabel allegoric loge lollipop salle collate prolix mast euphony1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 282 |
| ✓ | BbhWordSorting120CTSort the following words alphabetically: List: pea yonkers darkle quadrant moldboard instant olsen hookup erudite lip1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT528 |
| ✓ | BbhWordSorting121CTSort the following words alphabetically: List: regret uptake satisfy behold fumarole motif forth alcohol escutcheon pease hackberry walkie1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 285 |
| ✓ | BbhWordSorting122CTSort the following words alphabetically: List: charcuterie muddle diatom impenetrable footfall crucifix greenberg synchronous spoken1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 304 |
| ✓ | BbhWordSorting123CTSort the following words alphabetically: List: sibyl calligraph goat inverness form threadbare1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT662 |
| ✓ | BbhWordSorting124CTSort the following words alphabetically: List: cashew apprehension ensemble1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT381 |
| ✓ | BbhWordSorting125CTSort the following words alphabetically: List: seashore haughty1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT664 |
| ✓ | BbhWordSorting126CTSort the following words alphabetically: List: postcondition protoplasmic musicology helical uptrend vasoconstriction diacritic beefsteak beware birthplace bicycle junctor state obstinate banshee sap1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 582 |
| ✓ | BbhWordSorting127CTSort the following words alphabetically: List: whisper eyesight titmice mirage fletch cry biennial creedal miltonic fraudulent j1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT1 524 |
| ✓ | BbhWordSorting128CTSort the following words alphabetically: List: multitudinous adopt wage worcestershire pacifist afghan glimmer friday1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 020 |
| ✓ | BbhWordSorting129CTSort the following words alphabetically: List: lethe bilk perturb tactual1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT773 |
| ✓ | BbhWordSorting130CTSort the following words alphabetically: List: liggett sterno pintail dilate frolic piggish forth parameter slaughterhouse lacunae mcintosh bloodstain lazy unesco protector beachhead algonquin1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 381 |
| ✓ | BbhWordSorting131CTSort the following words alphabetically: List: erudite marriageable stahl coach inadvisable vicksburg virgo brownian eosine magnesium flax1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 612 |
| ✓ | BbhWordSorting132CTSort the following words alphabetically: List: clifford brindle siskin gloat florist triploidy willard sacramento1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT647 |
| × | BbhWordSorting133CTSort the following words alphabetically: List: confrontation quantitative proserpine proofread daddy hirsute1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("confrontation daddy hirsute proofread proserpine quantitative")
Context: BbhWordSorting133CT: expected sorted list, got [confrontation daddy hirsute proserpine proofread quantitative] |
Failed | IN973 OUT976 |
| ✓ | BbhWordSorting134CTSort the following words alphabetically: List: allyn carbonaceous majorca investigatory supersede cetacean pathogenic johann tung paradigmatic pray1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT788 |
| ✓ | BbhWordSorting135CTSort the following words alphabetically: List: calyx emphases vii elgin perquisite implacable blackbird afro jura mayapple whit blame1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 159 |
| ✓ | BbhWordSorting136CTSort the following words alphabetically: List: cast palazzi inhibit arragon tabletop danbury abbe adposition emplace gavin income falsetto onerous1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 162 |
| ✓ | BbhWordSorting137CTSort the following words alphabetically: List: ulcerate ballroom sparse intrepid arcing prorogue dodge benefit coupon ladle buggy shagging bassoon hypothermia decide affirmative airframe junior nineveh schmitt1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 310 |
| ✓ | BbhWordSorting138CTSort the following words alphabetically: List: statuette whiteface lauderdale encumber atonic specify olivia adipic neutrino antique catch persona athlete sovereignty1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 169 |
| ✓ | BbhWordSorting139CTSort the following words alphabetically: List: hyperboloidal borough1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT566 |
| × | BbhWordSorting140CTSort the following words alphabetically: List: yves butt scribe seaworthy manumitted clapeyron lift blackguard reconnaissance agile woodcut lightfooted sawbelly meager geophysical wiseacre mathieu cognoscenti purposive flamboyant1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("agile blackguard butt clapeyron cognoscenti flamboyant geophysical lift lightfooted manumitted mathieu meager purposive reconnaissance sawbelly scribe seaworthy wiseacre woodcut yves")
Context: BbhWordSorting140CT: expected sorted list, got [agile blackguard butt clapeyron cognoscenti flamboyant geophysical lightfooted lift manumitted mathieu meager purposive reconnaissance sawbelly scribe seaworthy wiseacre woodcut yves] |
Failed | IN1 005 OUT1 714 |
| ✓ | BbhWordSorting141CTSort the following words alphabetically: List: duane harriet impressible whippet glamour cowbell vietnamese fest switchboard anaglyph texture1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT982 |
| ✓ | BbhWordSorting142CTSort the following words alphabetically: List: plural dose allstate stalin dyad multitudinous powderpuff1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT828 |
| ✓ | BbhWordSorting143CTSort the following words alphabetically: List: crossbill bushland capacitive meager ricochet battery enigma wysiwyg wallet lipton jane wacke contingent1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 062 |
Reasoning - BBH word sorting (set 5)
26 passed1 failed |
96.3% |
IN26 493 OUT31 335 | |
| ✓ | BbhWordSorting144CTSort the following words alphabetically: List: corpulent diagnose code scalp bombproof fluorine blythe swipe honeybee damn pore maharaja cytolysis solicit1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 584 |
| ✓ | BbhWordSorting145CTSort the following words alphabetically: List: fugue chicanery mountain1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT607 |
| ✓ | BbhWordSorting146CTSort the following words alphabetically: List: sideboard procrustean seepage buxton olympic tty typescript marque callus unitary cameron contribute precise verify shelf extensible methanol1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 882 |
| ✓ | BbhWordSorting147CTSort the following words alphabetically: List: sociolinguistic nigeria odysseus demystify proton voltaire penny contaminate bighorn stirrup1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT929 |
| ✓ | BbhWordSorting148CTSort the following words alphabetically: List: greasy lapidary mark1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT550 |
| ✓ | BbhWordSorting149CTSort the following words alphabetically: List: choreograph retardant multifarious pro carbonic equestrian wishful wigwam countersink libya authenticate metal have corvallis nitric petition obfuscatory1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 467 |
| ✓ | BbhWordSorting150CTSort the following words alphabetically: List: oakland seaborg jacobi membrane trapezoidal allis marmot toggle anthology1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 287 |
| ✓ | BbhWordSorting151CTSort the following words alphabetically: List: monotonous slick provocation functor hideaway holeable assimilable lola calypso nebuchadnezzar dogwood pacifism bivariate bought1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 348 |
| ✓ | BbhWordSorting152CTSort the following words alphabetically: List: livre substitution ix frisian panoramic buckley1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT622 |
| ✓ | BbhWordSorting153CTSort the following words alphabetically: List: no spokane cartilaginous that'd science1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT1 421 |
| ✓ | BbhWordSorting154CTSort the following words alphabetically: List: splashy rectitude novelty1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT466 |
| ✓ | BbhWordSorting155CTSort the following words alphabetically: List: sediment varsity towel fracture1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT581 |
| ✓ | BbhWordSorting156CTSort the following words alphabetically: List: confidential ursa fiction heterozygous faery overture horehound1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT721 |
| ✓ | BbhWordSorting157CTSort the following words alphabetically: List: mckinley pesticide school radiometer marque prig relish trianon inspiration artful cancelled castrate glaucous ear croon tioga rothschild excite endpoint citadel1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 767 |
| ✓ | BbhWordSorting158CTSort the following words alphabetically: List: iniquity yoghurt behind volcanism hornpipe mcconnell scorn zip toroidal mollie sandy wellwisher inmate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 794 |
| ✓ | BbhWordSorting159CTSort the following words alphabetically: List: extempore gotten1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT596 |
| ✓ | BbhWordSorting160CTSort the following words alphabetically: List: yang relinquish merry maladjust olefin canaan ranch batavia1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT1 079 |
| ✓ | BbhWordSorting161CTSort the following words alphabetically: List: yuh edt cheddar oblivion from poignant pang1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT823 |
| ✓ | BbhWordSorting162CTSort the following words alphabetically: List: zan natalie coplanar stevenson1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT910 |
| ✓ | BbhWordSorting163CTSort the following words alphabetically: List: bottle kenney chapati okinawa boon bedtime1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT1 092 |
| × | BbhWordSorting164CTSort the following words alphabetically: List: contrariety hoop muong dimorphic crochet forthwith hurty titrate prodigy polytope guaranteeing atavism emanate iniquity grind more katie1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("atavism contrariety crochet dimorphic emanate forthwith grind guaranteeing hoop hurty iniquity katie more muong polytope prodigy titrate")
Context: BbhWordSorting164CT: expected sorted list, got [atavism, crochet, contrariety, dimorphic, emanate, forthwith, grind, guaranteeing, hoop, hurty, iniquity, katie, more, muong, polytope, prodigy, titrate] |
Failed | IN996 OUT1 984 |
| ✓ | BbhWordSorting165CTSort the following words alphabetically: List: sloane molten equip hillman max cochran practise advisee kuwait chancery hoofprint cardigan evildoer retinue wuhan ape apostate acrobacy crowbait1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT1 366 |
| ✓ | BbhWordSorting166CTSort the following words alphabetically: List: catawba almagest transfusable milt photolysis limpid gnome archenemy sagging councilwomen decrement jungian1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 035 |
| ✓ | BbhWordSorting167CTSort the following words alphabetically: List: inscribe upraise perceptive tragedian peculate cunard posterior crude danubian1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 700 |
| ✓ | BbhWordSorting168CTSort the following words alphabetically: List: madeira offprint trafficked immemorial subject micro clinic sprue marx va anaheim eaten1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT965 |
| ✓ | BbhWordSorting169CTSort the following words alphabetically: List: reedy bully multiple edt sell referring lexicographer tit flogging perceptive minor bosporus rancorous forfeit pizza pungent sedition cork1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 327 |
| ✓ | BbhWordSorting170CTSort the following words alphabetically: List: thirteenth tinfoil thimble snuff ernie pow celebrity abuilding indiscoverable chigger burgher synchronous yap medlar exorcism newsboy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT1 432 |
Reasoning - BBH word sorting (set 6)
24 passed3 failed |
88.9% |
IN26 640 OUT35 796 | |
| ✓ | BbhWordSorting171CTSort the following words alphabetically: List: saprophytic confident pam stowaway stupor foundling compton1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT991 |
| ✓ | BbhWordSorting172CTSort the following words alphabetically: List: epiphyte swift sail anchor barre dimension faunal integrate eleanor edgy singular patrician orthodoxy parasol pendant buckle concatenate masochist1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT1 070 |
| ✓ | BbhWordSorting173CTSort the following words alphabetically: List: butterscotch veterinarian kochab molybdate extoller amos farrell academia glance predatory beautiful ford fiducial metzler europium circuitous diatom monomer1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 505 |
| ✓ | BbhWordSorting174CTSort the following words alphabetically: List: mater rica sigmund dean eosine inebriate gibson vassar mulligatawny formula1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 034 |
| ✓ | BbhWordSorting175CTSort the following words alphabetically: List: shakespearian laudatory1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT646 |
| ✓ | BbhWordSorting176CTSort the following words alphabetically: List: fasciculate judicature presto1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT619 |
| × | BbhWordSorting177CTSort the following words alphabetically: List: thereupon wireman crag scrawny electrolytic wife cutover dickson clytemnestra took diocletian lipton inhuman stalk workplace marginal1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("clytemnestra crag cutover dickson diocletian electrolytic inhuman lipton marginal scrawny stalk thereupon took wife wireman workplace")
Context: BbhWordSorting177CT: expected sorted list, got [clytemnestra crag cutover dickson diocletian electrolytic inhuman lipton marginal scrawny stalk took thereupon wife wireman workplace] |
Failed | IN996 OUT1 354 |
| ✓ | BbhWordSorting178CTSort the following words alphabetically: List: bruckner admiration technion bandgap heron postprocessor maidenhair fungus allusion gambit accessory proscenium cruise1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT1 256 |
| × | BbhWordSorting179CTSort the following words alphabetically: List: grammarian chivalry polkadot gastronomic ferocity brenda knutson endothelial phenotype irreducible bilinear rockaway derivate eaten cacao scurrilous1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("bilinear brenda cacao chivalry derivate eaten endothelial ferocity gastronomic grammarian irreducible knutson phenotype polkadot rockaway scurrilous")
Context: BbhWordSorting179CT: expected sorted list, got [bilinear brenda cacao chivalry derivate eaten endothelial ferocity gastronomic grammarians irreducible knutson phenotype polkadot rockaway scurrilous] |
Failed | IN996 OUT2 090 |
| ✓ | BbhWordSorting180CTSort the following words alphabetically: List: hyperboloid buckle liquidate alkali nagasaki breach falsetto parmesan mirth1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT695 |
| × | BbhWordSorting181CTSort the following words alphabetically: List: hornblower dissipate amanita canticle annoy besiege straight notre propylene sepia california pasture encephalitis boggle crocodilian dexter snipe amatory dizzy psychiatric1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("amanita amatory annoy besiege boggle california canticle crocodilian dexter dissipate dizzy encephalitis hornblower notre pasture propylene psychiatric sepia snipe straight")
Context: BbhWordSorting181CT: expected sorted list, got [annoy amanita amatory besiege boggle california canticle crocodilian dexter dissipate dizzy encephalitis hornblower notre pasture psychiatric propylene sepia snipe straight] |
Failed | IN999 OUT1 408 |
| ✓ | BbhWordSorting182CTSort the following words alphabetically: List: peaceable residue sulfanilamide kingsley eft caliber headboard stamp phillip upholster irishman lobby nary gastrointestinal fragile faulkner capricious ouzo phylum1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 846 |
| ✓ | BbhWordSorting183CTSort the following words alphabetically: List: multiplet catsup hypothalamus scherzo bridesmaid average therewith invocable coddle charm inequity region abbas trumpery memorandum shutout integral dogfish phloem inconvertible1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 003 OUT1 785 |
| ✓ | BbhWordSorting184CTSort the following words alphabetically: List: windowsill appoint biharmonic moustache baneberry wiry dyne pirate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT1 135 |
| ✓ | BbhWordSorting185CTSort the following words alphabetically: List: zoroaster skimpy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT374 |
| ✓ | BbhWordSorting186CTSort the following words alphabetically: List: almost delphic semitic unbeknownst astute deadlock libertarian antic levitate execution sanction scathe storehouse dandelion affable sweeney fortunate leverage horntail apache1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT3 460 |
| ✓ | BbhWordSorting187CTSort the following words alphabetically: List: percent maser roadside we're caching distort phillip emboss epistemology gherkin somewhat spicy injustice winston savoyard delicious defend indicate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 708 |
| ✓ | BbhWordSorting188CTSort the following words alphabetically: List: anarchic liven shortcut total diatom guelph usage acoustic yarmulke phenomenology neo bureaucracy trammel leftward immovable fabricate syndrome suggestive provide1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT1 432 |
| ✓ | BbhWordSorting189CTSort the following words alphabetically: List: them abdominal address fomalhaut pay optimal hanoverian yang berry tattoo islamabad swathe jordan bounty stearic stigmata involve effusive tornado1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 673 |
| ✓ | BbhWordSorting190CTSort the following words alphabetically: List: maraud downtrodden hurst propane morphine parsonage gadgetry inertial gamin1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 717 |
| ✓ | BbhWordSorting191CTSort the following words alphabetically: List: foamflower lawful cayenne chandigarh only excursion declassify equanimity aseptic inclement october debauch kruger groupoid scorch dingy alterate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 677 |
| ✓ | BbhWordSorting192CTSort the following words alphabetically: List: acidify antagonism asteria1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT521 |
| ✓ | BbhWordSorting193CTSort the following words alphabetically: List: elton olympic iodinate captious ineligible sherman1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT637 |
| ✓ | BbhWordSorting194CTSort the following words alphabetically: List: alleviate duopoly mattress gland benelux townsmen buoyant klaxon hardbound tomography felice gunk1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT992 |
| ✓ | BbhWordSorting195CTSort the following words alphabetically: List: solvate nat usable kajar downey vision detest aitken barycentric1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT731 |
| ✓ | BbhWordSorting196CTSort the following words alphabetically: List: griffith common nanosecond loosestrife decoy swabby aerospace denmark yates frill spitz enviable screechy allure sow spermatozoa saute jibe exclusive1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 001 OUT2 383 |
| ✓ | BbhWordSorting197CTSort the following words alphabetically: List: reverie giantess muddy mast callous bate dnieper prank cortez staunch satisfy dogging moran climb garrison1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT1 057 |
Reasoning - BBH word sorting (set 7)
25 passed1 failed |
96.2% |
IN25 546 OUT30 301 | |
| ✓ | BbhWordSorting198CTSort the following words alphabetically: List: avoid whimsey grassland elegy pitfall error gaillardia hostile caramel conclave stepchild accept carbuncle compressor utopia drib spectra rosa embower1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 913 |
| ✓ | BbhWordSorting199CTSort the following words alphabetically: List: rowley midshipman saline furl geld locomote exhumation anarchic bstj gradual riddance j liniment pantheist elution profess1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 310 |
| ✓ | BbhWordSorting200CTSort the following words alphabetically: List: hangout drafty supposable sushi proscenium anticonvulsant you'd magnificent scrub embolden hasty pewee carrageen disseminate glamour discovery registrar acuity1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 995 |
| ✓ | BbhWordSorting201CTSort the following words alphabetically: List: sweatband murray1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT450 |
| ✓ | BbhWordSorting202CTSort the following words alphabetically: List: resignation dormouse bathos walt amethyst obtuse1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT713 |
| ✓ | BbhWordSorting203CTSort the following words alphabetically: List: paginate amicable they've histology extreme indoeuropean sonority tramway turvy coo deerstalker browne henchman pelvis calumny1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT1 399 |
| ✓ | BbhWordSorting204CTSort the following words alphabetically: List: transgressor sidewinder tacky casualty shouldn't courtier leprosy avoidance gibbon merge1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 495 |
| ✓ | BbhWordSorting205CTSort the following words alphabetically: List: intercept tamp geopolitic loci newsmen umber credit atmospheric lunge siren swart chess1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT1 238 |
| ✓ | BbhWordSorting206CTSort the following words alphabetically: List: wink envious scotia planetaria pooh emancipate army1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT881 |
| ✓ | BbhWordSorting207CTSort the following words alphabetically: List: condescend rampant percolate coltish date rochester placid significant1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 426 |
| ✓ | BbhWordSorting208CTSort the following words alphabetically: List: infantrymen knob yaounde aqueous deregulate lysergic gala1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT614 |
| ✓ | BbhWordSorting209CTSort the following words alphabetically: List: promulgate altercate foraminifera sophocles raft wrongdoer syllabus jive cornerstone gossamer courtroom insist dusenberg sal1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT1 708 |
| ✓ | BbhWordSorting210CTSort the following words alphabetically: List: jacob sleight globule consonant musician1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT721 |
| ✓ | BbhWordSorting211CTSort the following words alphabetically: List: nascent muddy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN965 OUT508 |
| ✓ | BbhWordSorting212CTSort the following words alphabetically: List: automate abner shampoo gurgle none exquisite abramson amity fruitful shorten waterproof1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 047 |
| ✓ | BbhWordSorting213CTSort the following words alphabetically: List: search marcello depreciate unary frame numbly sprout clench agamemnon herbivorous eject zaire lien forum1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT1 869 |
| ✓ | BbhWordSorting214CTSort the following words alphabetically: List: saul fresco teen chopstick footwork smalley anaplasmosis palisade orthography storey bumble pilate clue ingot fiesta1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 460 |
| ✓ | BbhWordSorting215CTSort the following words alphabetically: List: avalanche chaplin squabble coachman ipsilateral clonic cameroon fortran cram medea referring kennan canal pyridine ussr postpone1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT1 287 |
| × | BbhWordSorting216CTSort the following words alphabetically: List: peculiar margaret contravention ingratiate dreg superintendent drapery bizarre sequential1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("bizarre contravention drapery dreg ingratiate margaret peculiar sequential superintendent")
Context: BbhWordSorting216CT: expected sorted list, got [bizarre, contravention, drapery, dreg, ingrati ate, margaret, peculiar, sequential, superintendent] |
Failed | IN979 OUT1 589 |
| ✓ | BbhWordSorting217CTSort the following words alphabetically: List: sweatshirt bone ia wagoneer doleful homeobox hindustan convergent1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT925 |
| ✓ | BbhWordSorting218CTSort the following words alphabetically: List: pewter gross gujarati cortland diffusible nomograph stag broadcast galvanometer transition incestuous sketchbook scout larynx1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 249 |
| ✓ | BbhWordSorting219CTSort the following words alphabetically: List: portsmouth impalpable portrait gouda brainy succumb fatuous hoof lisbon erudite malaria enigma isaacson cony stronghold servomechanism1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 423 |
| ✓ | BbhWordSorting220CTSort the following words alphabetically: List: testate orchestra permafrost leasehold shiva1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT660 |
| ✓ | BbhWordSorting221CTSort the following words alphabetically: List: logarithm robertson inanimate uganda micron incurring wilcox lumber sammy souvenir newman profuse maladapt cone butch airlift homeowner1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT850 |
| ✓ | BbhWordSorting222CTSort the following words alphabetically: List: chrysalis wallaby1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT550 |
| ✓ | BbhWordSorting223CTSort the following words alphabetically: List: dredge checksum huckster baronial spotlight circumstance eulogy comment felicia dartmouth monochromator neuroanatomic emittance1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT1 021 |
Reasoning - BBH word sorting (set 8)
26 passed |
100.0% |
IN25 604 OUT30 297 | |
| ✓ | BbhWordSorting224CTSort the following words alphabetically: List: breezy germinal biplane canada nutrient archipelago peloponnese thigh conspiracy constructor accrue dobbin lost hamburger insubstantial malleable ted laramie1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 220 |
| ✓ | BbhWordSorting225CTSort the following words alphabetically: List: wagging cabdriver astronomic pivot loch coherent1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT920 |
| ✓ | BbhWordSorting226CTSort the following words alphabetically: List: inability nucleus vague possible invocable janice erg1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT660 |
| ✓ | BbhWordSorting227CTSort the following words alphabetically: List: gag grecian ada nobody militia wildcat right abc blend cankerworm hanukkah pierson ron falcon quad flamboyant lobster kruger austere indicate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT1 452 |
| ✓ | BbhWordSorting228CTSort the following words alphabetically: List: babysat curvaceous hugh cutaneous spoke consul stationarity regiment1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT1 327 |
| ✓ | BbhWordSorting229CTSort the following words alphabetically: List: woebegone nimble complementary afternoon immaculate laughlin kind hesitate loire septuagenarian sandia toggle dixie horsepower sierra shuffleboard mechanism1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 320 |
| ✓ | BbhWordSorting230CTSort the following words alphabetically: List: linchpin arrear eavesdropping fasciculate shin postdoctoral maritime ibn henry hermaphrodite sticky jane brookside incorrigible vehicular herodotus1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 001 OUT1 158 |
| ✓ | BbhWordSorting231CTSort the following words alphabetically: List: sinter cortex kangaroo pillage threshold insane mcleod roundabout stipulate marionette trammel incident1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 196 |
| ✓ | BbhWordSorting232CTSort the following words alphabetically: List: soulful dnieper labile lease vehicular1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT848 |
| ✓ | BbhWordSorting233CTSort the following words alphabetically: List: cornerstone green brindle prowl nonce ballard scrappy credulous des pitney difluoride toward predilect nostalgic rape mixup horseplay jew curio1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 320 |
| ✓ | BbhWordSorting234CTSort the following words alphabetically: List: covenant thence davenport sonorant scoreboard densitometer noisy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT759 |
| ✓ | BbhWordSorting235CTSort the following words alphabetically: List: inhabitation bona cicero purport gaussian silver responsible rastus zig exonerate portfolio coastal asset ruanda dusky handlebar1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 208 |
| ✓ | BbhWordSorting236CTSort the following words alphabetically: List: implementor bangui nudge cavalier refractory islamabad contextual abramson waite railway carlisle magistrate silvery picnicking dustbin emacs1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 198 |
| ✓ | BbhWordSorting237CTSort the following words alphabetically: List: strove hendrickson apostasy jacm indonesia vibrate afloat lanthanide grosbeak scavenge quark chattel conner bechtel melancholy ferment1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT1 298 |
| ✓ | BbhWordSorting238CTSort the following words alphabetically: List: spongy separate gabriel jackson quail hendrickson washy backslide shelter calvert anastomosis variety commando pizzicato sticktight syndicate hollister1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 831 |
| ✓ | BbhWordSorting239CTSort the following words alphabetically: List: behold oxalic maybe hew steel termcap pray stiffen dissipate misogyny format dew1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT1 206 |
| ✓ | BbhWordSorting240CTSort the following words alphabetically: List: dove rodriguez sonant monaco strap horticulture claret play paschal bizarre valuate colonist baden deplore1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 276 |
| ✓ | BbhWordSorting241CTSort the following words alphabetically: List: patrolman arabesque novel spearmint arkansan bend brassiere diagnosable ytterbium sheep bedroom yen planeload deterrent fossiliferous alveolar trident fluke curvilinear1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 002 OUT1 872 |
| ✓ | BbhWordSorting242CTSort the following words alphabetically: List: papaw perfidious frugal labial feel sonar blunderbuss box dinnertime oresteia1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 556 |
| ✓ | BbhWordSorting243CTSort the following words alphabetically: List: haddock colossus sheepskin honeymoon candid pragmatic agrarian people applicate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT668 |
| ✓ | BbhWordSorting244CTSort the following words alphabetically: List: catechism daddy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT543 |
| ✓ | BbhWordSorting245CTSort the following words alphabetically: List: holocene muriel r&d pathetic talon mare staircase bradshaw aperture sigh1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT998 |
| ✓ | BbhWordSorting246CTSort the following words alphabetically: List: bust midshipmen musicale urinary chalk rocket resiny horror tangential sapient dumb serf pathway urea goucher fatty masonry dentistry cowboy roadrunner1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 655 |
| ✓ | BbhWordSorting247CTSort the following words alphabetically: List: sanhedrin scratchy helical beau venezuela awash bessie extricable indoeuropean vice pendulum cream animism1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 156 |
| ✓ | BbhWordSorting248CTSort the following words alphabetically: List: him correspond seashore herpes1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT785 |
| ✓ | BbhWordSorting249CTSort the following words alphabetically: List: worcestershire bonito fritter dreamboat haggard nose whodunit1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT867 |