Benchmark summary
BBH-based reasoning benchmark ported to the SC# evaluation runtime. It tests multi-step reasoning across challenging tasks involving logic, deduction, symbolic manipulation, planning, and problem solving.
Tests with multiple category tags are counted in each matching category, so category totals may exceed the report total.
| Benchmark items | |||
|---|---|---|---|
| Script / test | Status | Tokens | |
Reasoning - BBH boolean expressions (set 1)
32 passed |
100.0% |
IN30 848 OUT11 362 | |
| ✓ | BbhBooleanExpressionsBD1not True or ( False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT408 |
| ✓ | BbhBooleanExpressionsBD10( True and not True and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT411 |
| ✓ | BbhBooleanExpressionsBD11False and False and False or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT324 |
| ✓ | BbhBooleanExpressionsBD12False or ( False and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT258 |
| ✓ | BbhBooleanExpressionsBD13True and False or ( not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT361 |
| ✓ | BbhBooleanExpressionsBD14not not ( True ) and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT296 |
| ✓ | BbhBooleanExpressionsBD15not False or ( True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT275 |
| ✓ | BbhBooleanExpressionsBD16not ( True ) or False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT209 |
| ✓ | BbhBooleanExpressionsBD17( True and not not not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT201 |
| ✓ | BbhBooleanExpressionsBD18( False or not False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT307 |
| ✓ | BbhBooleanExpressionsBD19False and False or True and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT303 |
| ✓ | BbhBooleanExpressionsBD2not True or ( True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT322 |
| ✓ | BbhBooleanExpressionsBD20not not False or not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT326 |
| ✓ | BbhBooleanExpressionsBD21True and not True or False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT297 |
| ✓ | BbhBooleanExpressionsBD22not False or True and False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT285 |
| ✓ | BbhBooleanExpressionsBD23False and ( not False ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT378 |
| ✓ | BbhBooleanExpressionsBD24not not not not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT734 |
| ✓ | BbhBooleanExpressionsBD25( True ) and True or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT328 |
| ✓ | BbhBooleanExpressionsBD26not True or False and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT295 |
| ✓ | BbhBooleanExpressionsBD27not not not not not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT331 |
| ✓ | BbhBooleanExpressionsBD28not True or not False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT282 |
| ✓ | BbhBooleanExpressionsBD29False or True or False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT1 174 |
| ✓ | BbhBooleanExpressionsBD3False or ( False ) and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT326 |
| ✓ | BbhBooleanExpressionsBD30not ( True or True ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT348 |
| ✓ | BbhBooleanExpressionsBD31not not not not True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT412 |
| ✓ | BbhBooleanExpressionsBD32( False ) or False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT311 |
| ✓ | BbhBooleanExpressionsBD4not False or True and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT392 |
| ✓ | BbhBooleanExpressionsBD5not True or False or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT285 |
| ✓ | BbhBooleanExpressionsBD6True and True and False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT307 |
| ✓ | BbhBooleanExpressionsBD7not not True and not True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT305 |
| ✓ | BbhBooleanExpressionsBD8not not not ( True and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT244 |
| ✓ | BbhBooleanExpressionsBD9not not False and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT327 |
Reasoning - BBH boolean expressions (set 2)
31 passed1 failed |
96.9% |
IN30 848 OUT8 861 | |
| ✓ | BbhBooleanExpressionsBD33( False and True or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT297 |
| ✓ | BbhBooleanExpressionsBD34not not not ( False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT226 |
| ✓ | BbhBooleanExpressionsBD35not not True and not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT286 |
| ✓ | BbhBooleanExpressionsBD36not False or True and not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT304 |
| ✓ | BbhBooleanExpressionsBD37True or True or not False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT236 |
| ✓ | BbhBooleanExpressionsBD38( not False and False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT205 |
| ✓ | BbhBooleanExpressionsBD39( ( not True ) or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT170 |
| ✓ | BbhBooleanExpressionsBD40not not ( not False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT258 |
| ✓ | BbhBooleanExpressionsBD41not True or False or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT277 |
| ✓ | BbhBooleanExpressionsBD42False or ( not False ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT280 |
| ✓ | BbhBooleanExpressionsBD43not False or False or ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT245 |
| ✓ | BbhBooleanExpressionsBD44not not False or False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT314 |
| ✓ | BbhBooleanExpressionsBD45True or ( ( not True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT163 |
| ✓ | BbhBooleanExpressionsBD46not False and not False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT200 |
| ✓ | BbhBooleanExpressionsBD47( False ) and not False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT493 |
| ✓ | BbhBooleanExpressionsBD48( True ) and False or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT269 |
| ✓ | BbhBooleanExpressionsBD49( not False and ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT227 |
| ✓ | BbhBooleanExpressionsBD50False or not ( False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT319 |
| ✓ | BbhBooleanExpressionsBD51True and not False and not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT355 |
| ✓ | BbhBooleanExpressionsBD52not True and True and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT308 |
| ✓ | BbhBooleanExpressionsBD53not not ( False and not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT311 |
| × | BbhBooleanExpressionsBD54True or not ( ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("True")
Context: BbhBooleanExpressionsBD54: expected True, got [not]
Values:
got = "not" |
Failed | IN964 OUT348 |
| ✓ | BbhBooleanExpressionsBD55False or not True or False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT262 |
| ✓ | BbhBooleanExpressionsBD56False or not not True or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT325 |
| ✓ | BbhBooleanExpressionsBD57not ( True or True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT219 |
| ✓ | BbhBooleanExpressionsBD58False or not not not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT327 |
| ✓ | BbhBooleanExpressionsBD59not ( False ) or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT306 |
| ✓ | BbhBooleanExpressionsBD60True and not False or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT303 |
| ✓ | BbhBooleanExpressionsBD61True or ( not ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT170 |
| ✓ | BbhBooleanExpressionsBD62( False or True and not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT260 |
| ✓ | BbhBooleanExpressionsBD63False or not True and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT294 |
| ✓ | BbhBooleanExpressionsBD64not not not not not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT304 |
Reasoning - BBH boolean expressions (set 3)
32 passed |
100.0% |
IN30 848 OUT9 489 | |
| ✓ | BbhBooleanExpressionsBD65( not False ) and False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT219 |
| ✓ | BbhBooleanExpressionsBD66False and False or not ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT248 |
| ✓ | BbhBooleanExpressionsBD67( not False or True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT507 |
| ✓ | BbhBooleanExpressionsBD68not ( False ) and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT188 |
| ✓ | BbhBooleanExpressionsBD69False or True and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT315 |
| ✓ | BbhBooleanExpressionsBD70( True ) and True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT385 |
| ✓ | BbhBooleanExpressionsBD71( not True ) and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT246 |
| ✓ | BbhBooleanExpressionsBD72( not not True and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT305 |
| ✓ | BbhBooleanExpressionsBD73True and ( False ) or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT314 |
| ✓ | BbhBooleanExpressionsBD74not not not False or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT347 |
| ✓ | BbhBooleanExpressionsBD75not not not False and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT389 |
| ✓ | BbhBooleanExpressionsBD76True or ( True or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT267 |
| ✓ | BbhBooleanExpressionsBD77not False or ( ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT314 |
| ✓ | BbhBooleanExpressionsBD78not not ( False and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT369 |
| ✓ | BbhBooleanExpressionsBD79not not not not True or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT319 |
| ✓ | BbhBooleanExpressionsBD80False and not ( False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT236 |
| ✓ | BbhBooleanExpressionsBD81True and True and True or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT304 |
| ✓ | BbhBooleanExpressionsBD82False or not False or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT337 |
| ✓ | BbhBooleanExpressionsBD83not False and False and False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT254 |
| ✓ | BbhBooleanExpressionsBD84( False ) or not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT272 |
| ✓ | BbhBooleanExpressionsBD85( not False ) or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT169 |
| ✓ | BbhBooleanExpressionsBD86not not ( False or not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT326 |
| ✓ | BbhBooleanExpressionsBD87False or not False or False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT313 |
| ✓ | BbhBooleanExpressionsBD88False or not False and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT277 |
| ✓ | BbhBooleanExpressionsBD89False or ( False ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT229 |
| ✓ | BbhBooleanExpressionsBD90not True or False and True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT313 |
| ✓ | BbhBooleanExpressionsBD91True or ( True or not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT262 |
| ✓ | BbhBooleanExpressionsBD92True and not False and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT397 |
| ✓ | BbhBooleanExpressionsBD93False or not False or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT350 |
| ✓ | BbhBooleanExpressionsBD94( True or not False ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT192 |
| ✓ | BbhBooleanExpressionsBD95False or not ( True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT270 |
| ✓ | BbhBooleanExpressionsBD96not False and not True and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT256 |
Reasoning - BBH boolean expressions (set 4)
30 passed2 failed |
93.8% |
IN30 848 OUT10 226 | |
| ✓ | BbhBooleanExpressionsBD100( True or not not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT259 |
| ✓ | BbhBooleanExpressionsBD101False and ( False or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT270 |
| ✓ | BbhBooleanExpressionsBD102False or not True or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT306 |
| ✓ | BbhBooleanExpressionsBD103True and not True or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT341 |
| × | BbhBooleanExpressionsBD104( not False or ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("True")
Context: BbhBooleanExpressionsBD104: expected True, got [false]
Values:
got = "false" |
Failed | IN964 OUT207 |
| ✓ | BbhBooleanExpressionsBD105( ( not ( True ) ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT248 |
| ✓ | BbhBooleanExpressionsBD106False and False and not ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT309 |
| ✓ | BbhBooleanExpressionsBD107not True and not True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT246 |
| ✓ | BbhBooleanExpressionsBD108not ( True or False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT346 |
| ✓ | BbhBooleanExpressionsBD109False or ( True ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT264 |
| ✓ | BbhBooleanExpressionsBD110False or False and False or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT367 |
| × | BbhBooleanExpressionsBD111True or False or not True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("True")
Context: BbhBooleanExpressionsBD111: expected True, got [false]
Values:
got = "false" |
Failed | IN964 OUT1 446 |
| ✓ | BbhBooleanExpressionsBD112not True and ( True and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT283 |
| ✓ | BbhBooleanExpressionsBD113not True and True or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT323 |
| ✓ | BbhBooleanExpressionsBD114not False or ( True or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT237 |
| ✓ | BbhBooleanExpressionsBD115not ( False or ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT189 |
| ✓ | BbhBooleanExpressionsBD116not ( True or True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT311 |
| ✓ | BbhBooleanExpressionsBD117False or not False or True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT293 |
| ✓ | BbhBooleanExpressionsBD118False or False and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT358 |
| ✓ | BbhBooleanExpressionsBD119not ( True or False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT200 |
| ✓ | BbhBooleanExpressionsBD120( False ) or not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT272 |
| ✓ | BbhBooleanExpressionsBD121True and True and not not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT361 |
| ✓ | BbhBooleanExpressionsBD122( True ) or True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT287 |
| ✓ | BbhBooleanExpressionsBD123False and False and not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT254 |
| ✓ | BbhBooleanExpressionsBD124( True ) and not True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT265 |
| ✓ | BbhBooleanExpressionsBD125not not not True and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT398 |
| ✓ | BbhBooleanExpressionsBD126False and False and not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT422 |
| ✓ | BbhBooleanExpressionsBD127False and False or False or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT309 |
| ✓ | BbhBooleanExpressionsBD128False or ( not ( False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT171 |
| ✓ | BbhBooleanExpressionsBD97False and ( True and not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT205 |
| ✓ | BbhBooleanExpressionsBD98not False or ( False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT249 |
| ✓ | BbhBooleanExpressionsBD99False and not ( False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT230 |
Reasoning - BBH boolean expressions (set 5)
30 passed |
100.0% |
IN28 920 OUT8 307 | |
| ✓ | BbhBooleanExpressionsBD129not True and True and not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT349 |
| ✓ | BbhBooleanExpressionsBD130( False ) and ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT191 |
| ✓ | BbhBooleanExpressionsBD131( not True ) and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT277 |
| ✓ | BbhBooleanExpressionsBD132False and True and ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT244 |
| ✓ | BbhBooleanExpressionsBD133( ( True ) ) or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT185 |
| ✓ | BbhBooleanExpressionsBD134not True and False or False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT336 |
| ✓ | BbhBooleanExpressionsBD135True and not True and True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT379 |
| ✓ | BbhBooleanExpressionsBD136( True or True or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT165 |
| ✓ | BbhBooleanExpressionsBD137True or not False and True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT243 |
| ✓ | BbhBooleanExpressionsBD138( not ( False ) or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT217 |
| ✓ | BbhBooleanExpressionsBD139True and not ( True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT250 |
| ✓ | BbhBooleanExpressionsBD140not ( False and True ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT253 |
| ✓ | BbhBooleanExpressionsBD141not ( True and True ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT334 |
| ✓ | BbhBooleanExpressionsBD142False or not True and not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT301 |
| ✓ | BbhBooleanExpressionsBD143not True and False or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT274 |
| ✓ | BbhBooleanExpressionsBD144not True or True and False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT360 |
| ✓ | BbhBooleanExpressionsBD145( not ( False ) and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT228 |
| ✓ | BbhBooleanExpressionsBD146False or ( True and not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT233 |
| ✓ | BbhBooleanExpressionsBD147not False and not True or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT245 |
| ✓ | BbhBooleanExpressionsBD148( False or not not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT403 |
| ✓ | BbhBooleanExpressionsBD149( not False ) or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT252 |
| ✓ | BbhBooleanExpressionsBD150( not not True ) or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT230 |
| ✓ | BbhBooleanExpressionsBD151True or not ( ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT254 |
| ✓ | BbhBooleanExpressionsBD152( False ) and False or not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT452 |
| ✓ | BbhBooleanExpressionsBD153not False and True or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT199 |
| ✓ | BbhBooleanExpressionsBD154True or not not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT263 |
| ✓ | BbhBooleanExpressionsBD155False and not True and False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT293 |
| ✓ | BbhBooleanExpressionsBD156not not True and True and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT405 |
| ✓ | BbhBooleanExpressionsBD157not False or not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT260 |
| ✓ | BbhBooleanExpressionsBD158not not ( True ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT232 |
Reasoning - BBH boolean expressions (set 6)
31 passed1 failed |
96.9% |
IN30 848 OUT10 341 | |
| ✓ | BbhBooleanExpressionsBe166( True and not True and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT327 |
| ✓ | BbhBooleanExpressionsBe167True or False or not True or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT726 |
| ✓ | BbhBooleanExpressionsBe168not True or True and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT379 |
| ✓ | BbhBooleanExpressionsBe169True and not not not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT439 |
| ✓ | BbhBooleanExpressionsBe170not True and ( False ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT331 |
| ✓ | BbhBooleanExpressionsBe171False and False or not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT335 |
| ✓ | BbhBooleanExpressionsBe172True or False and not True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT514 |
| ✓ | BbhBooleanExpressionsBe173not ( False or False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT228 |
| ✓ | BbhBooleanExpressionsBe174not False or True or True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT323 |
| ✓ | BbhBooleanExpressionsBe175not False and False or False or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT288 |
| ✓ | BbhBooleanExpressionsBe176( not True ) and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT170 |
| ✓ | BbhBooleanExpressionsBe177True and not not False and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT363 |
| × | BbhBooleanExpressionsBe178not ( not not False ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("True")
Context: BbhBooleanExpressionsBe178: expected True, got [False] |
Failed | IN964 OUT262 |
| ✓ | BbhBooleanExpressionsBe179not ( not False ) and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT208 |
| ✓ | BbhBooleanExpressionsBe180False or True and not False or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT293 |
| ✓ | BbhBooleanExpressionsBe181( not not not True and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT269 |
| ✓ | BbhBooleanExpressionsBe182False and not ( True ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT306 |
| ✓ | BbhBooleanExpressionsBe183( not True and False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT313 |
| ✓ | BbhBooleanExpressionsBe184not ( not not True ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT278 |
| ✓ | BbhBooleanExpressionsBe185( not False ) or ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT176 |
| ✓ | BbhBooleanExpressionsBe186not not not False and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT394 |
| ✓ | BbhBooleanExpressionsBe187True or not True or not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT473 |
| ✓ | BbhBooleanExpressionsBe188not False or not not False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT415 |
| ✓ | BbhBooleanExpressionsBe189not False and ( False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT245 |
| ✓ | BbhBooleanExpressionsBe190False and True or ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT336 |
| ✓ | BbhBooleanExpressionsBe191False and not False and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT282 |
| ✓ | BbhBooleanExpressionsBe192not ( ( False and True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT195 |
| ✓ | BbhBooleanExpressionsBe193False or ( not False ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT218 |
| ✓ | BbhBooleanExpressionsBe194False and not False or False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT408 |
| ✓ | BbhBooleanExpressionsBe195not False and not not True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT247 |
| ✓ | BbhBooleanExpressionsBe196True and True or False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT312 |
| ✓ | BbhBooleanExpressionsBe197not True or False and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT288 |
Reasoning - BBH boolean expressions (set 7)
32 passed |
100.0% |
IN30 848 OUT11 615 | |
| ✓ | BbhBooleanExpressionsBe198True or False or not True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT1 001 |
| ✓ | BbhBooleanExpressionsBe199False or False or not True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT294 |
| ✓ | BbhBooleanExpressionsBe200False or False or True and not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT317 |
| ✓ | BbhBooleanExpressionsBe201( not ( True or False ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT189 |
| ✓ | BbhBooleanExpressionsBe202not False and False or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT333 |
| ✓ | BbhBooleanExpressionsBe203True and False or ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT250 |
| ✓ | BbhBooleanExpressionsBe204not False or not not not not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT440 |
| ✓ | BbhBooleanExpressionsBe205True and ( not ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT180 |
| ✓ | BbhBooleanExpressionsBe206True and False and not True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT286 |
| ✓ | BbhBooleanExpressionsBe207( ( not False ) ) and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT195 |
| ✓ | BbhBooleanExpressionsBe208not ( True ) and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT221 |
| ✓ | BbhBooleanExpressionsBe209True or not False and False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT251 |
| ✓ | BbhBooleanExpressionsBe210True and not ( False or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT244 |
| ✓ | BbhBooleanExpressionsBe211not True and False or False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT347 |
| ✓ | BbhBooleanExpressionsBe212not True or not False and not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT246 |
| ✓ | BbhBooleanExpressionsBe213not True or False or True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT324 |
| ✓ | BbhBooleanExpressionsBe214not True and True and False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT402 |
| ✓ | BbhBooleanExpressionsBe215False and False or not True and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT363 |
| ✓ | BbhBooleanExpressionsBe216( not False or False and True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT278 |
| ✓ | BbhBooleanExpressionsBe217( True or False ) or not True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT1 230 |
| ✓ | BbhBooleanExpressionsBe218not not not ( False ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT491 |
| ✓ | BbhBooleanExpressionsBe219not False and False and False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT301 |
| ✓ | BbhBooleanExpressionsBe220False and not True and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT309 |
| ✓ | BbhBooleanExpressionsBe221( False or not False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT285 |
| ✓ | BbhBooleanExpressionsBe222False or ( False or not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT268 |
| ✓ | BbhBooleanExpressionsBe223not False or True or False and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT263 |
| ✓ | BbhBooleanExpressionsBe224True or ( not False ) or True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT242 |
| ✓ | BbhBooleanExpressionsBe225not False and False or True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT292 |
| ✓ | BbhBooleanExpressionsBe226False and not ( not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT271 |
| ✓ | BbhBooleanExpressionsBe227not not ( not True ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT460 |
| ✓ | BbhBooleanExpressionsBe228False and not True and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT315 |
| ✓ | BbhBooleanExpressionsBe229True or False and not ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT727 |
Reasoning - BBH boolean expressions (set 8)
30 passed2 failed |
93.8% |
IN33 175 OUT16 383 | |
| ✓ | BbhBooleanExpressionsBe230not ( False ) and ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT173 |
| ✓ | BbhBooleanExpressionsBe231False and not ( ( True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT247 |
| ✓ | BbhBooleanExpressionsBe232not False or ( not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT281 |
| ✓ | BbhBooleanExpressionsBe233( ( not False ) and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT174 |
| ✓ | BbhBooleanExpressionsBe234True or False and not False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT364 |
| ✓ | BbhBooleanExpressionsBe235( ( not True and True ) ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT237 |
| ✓ | BbhBooleanExpressionsBe236not ( False ) or not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT376 |
| ✓ | BbhBooleanExpressionsBe237not ( False or False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT279 |
| ✓ | BbhBooleanExpressionsBe238False and not ( not not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT226 |
| ✓ | BbhBooleanExpressionsBe239False and False and not ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT243 |
| ✓ | BbhBooleanExpressionsBe240False or ( not False and False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT253 |
| ✓ | BbhBooleanExpressionsBe241False or ( not not not True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT400 |
| ✓ | BbhBooleanExpressionsBe242( not False ) or not not False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT279 |
| ✓ | BbhBooleanExpressionsBe243not False and False and True or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT283 |
| ✓ | BbhBooleanExpressionsBe244True or not False and ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT203 |
| ✓ | BbhBooleanExpressionsBe245( not ( False ) ) or False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT195 |
| ✓ | BbhBooleanExpressionsBe246True and not True or False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT259 |
| ✓ | BbhBooleanExpressionsBe247not ( not not not not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT448 |
| ✓ | BbhBooleanExpressionsBe248True and ( not True or False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT437 |
| ✓ | BbhBooleanExpressionsBe249( not not not False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT291 |
| × | BbhCausalJudgementBe10judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("Yes")
Context: BbhCausalJudgementBe10: expected Yes, got [No] |
Failed | IN1 113 OUT999 |
| ✓ | BbhCausalJudgementBe11judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 545 |
| ✓ | BbhCausalJudgementBe12judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT884 |
| ✓ | BbhCausalJudgementBe13judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 168 OUT548 |
| × | BbhCausalJudgementBe14judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("Yes")
Context: BbhCausalJudgementBe14: expected Yes, got [No] |
Failed | IN1 147 OUT1 432 |
| ✓ | BbhCausalJudgementBe15judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 182 OUT532 |
| ✓ | BbhCausalJudgementBe16judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT601 |
| ✓ | BbhCausalJudgementBe17judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 172 OUT1 978 |
| ✓ | BbhCausalJudgementBe18judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 353 OUT1 072 |
| ✓ | BbhCausalJudgementBe19judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT412 |
| ✓ | BbhCausalJudgementBe8judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT271 |
| ✓ | BbhCausalJudgementBe9judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT461 |
Reasoning - BBH Dyck languages - close the brackets (set 1)
24 passed6 failed |
80.0% |
IN29 771 OUT45 330 | |
| ✓ | BbhDyckLanguagesBi13correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT450 |
| ✓ | BbhDyckLanguagesBi14correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 009 OUT4 129 |
| × | BbhDyckLanguagesBi15correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> } >")
Context: BbhDyckLanguagesBi15: expected > } >, got [}>] |
Failed | IN984 OUT709 |
| ✓ | BbhDyckLanguagesBi16correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 869 |
| ✓ | BbhDyckLanguagesBi17correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT487 |
| ✓ | BbhDyckLanguagesBi18correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT482 |
| ✓ | BbhDyckLanguagesBi19correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT834 |
| ✓ | BbhDyckLanguagesBi20correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 064 OUT3 490 |
| ✓ | BbhDyckLanguagesBi21correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT329 |
| ✓ | BbhDyckLanguagesBi22correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 026 OUT7 513 |
| × | BbhDyckLanguagesBi23correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] > )")
Context: BbhDyckLanguagesBi23: expected ] > ), got [])>)] |
Failed | IN978 OUT437 |
| ✓ | BbhDyckLanguagesBi24correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT1 004 |
| ✓ | BbhDyckLanguagesBi25correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT616 |
| ✓ | BbhDyckLanguagesBi26correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT350 |
| ✓ | BbhDyckLanguagesBi27correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT1 689 |
| ✓ | BbhDyckLanguagesBi28correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT289 |
| × | BbhDyckLanguagesBi29correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] ] ]")
Context: BbhDyckLanguagesBi29: expected ] ] ], got [}]]]] |
Failed | IN994 OUT755 |
| ✓ | BbhDyckLanguagesBi30correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT1 042 |
| ✓ | BbhDyckLanguagesBi31correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 013 OUT3 286 |
| ✓ | BbhDyckLanguagesBi32correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT615 |
| ✓ | BbhDyckLanguagesBi33correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 022 OUT1 728 |
| ✓ | BbhDyckLanguagesBi34correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT317 |
| × | BbhDyckLanguagesBi35correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("}")
Context: BbhDyckLanguagesBi35: expected }, got [{ { { } } }] |
Failed | IN978 OUT614 |
| ✓ | BbhDyckLanguagesBi36correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 481 |
| × | BbhDyckLanguagesBi37correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") } >")
Context: BbhDyckLanguagesBi37: expected ) } >, got [< { ( ( ) ) } >] |
Failed | IN978 OUT1 207 |
| ✓ | BbhDyckLanguagesBi38correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT852 |
| × | BbhDyckLanguagesBi39correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} ) )")
Context: BbhDyckLanguagesBi39: expected } ) ), got [))] |
Failed | IN1 056 OUT7 193 |
| ✓ | BbhDyckLanguagesBi40correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT519 |
| ✓ | BbhDyckLanguagesBi41correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT590 |
| ✓ | BbhDyckLanguagesBi42correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT454 |
Reasoning - BBH Dyck languages - close the brackets (set 2)
30 passed2 failed |
93.8% |
IN31 990 OUT73 670 | |
| ✓ | BbhDyckLanguagesBj51correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT905 |
| × | BbhDyckLanguagesBj52correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") > >")
Context: BbhDyckLanguagesBj52: expected ) > >, got [)>] |
Failed | IN978 OUT1 081 |
| ✓ | BbhDyckLanguagesBj53correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT627 |
| × | BbhDyckLanguagesBj54correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBj54: expected ), got []]]>>}]}}>)] |
Failed | IN1 066 OUT9 150 |
| ✓ | BbhDyckLanguagesBj55correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT1 338 |
| ✓ | BbhDyckLanguagesBj56correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT388 |
| ✓ | BbhDyckLanguagesBj57correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT508 |
| ✓ | BbhDyckLanguagesBj58correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT986 |
| ✓ | BbhDyckLanguagesBj59correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT545 |
| ✓ | BbhDyckLanguagesBj60correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT973 |
| ✓ | BbhDyckLanguagesBj61correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 010 OUT1 052 |
| ✓ | BbhDyckLanguagesBj62correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT1 969 |
| ✓ | BbhDyckLanguagesBj63correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT997 |
| ✓ | BbhDyckLanguagesBj64correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT901 |
| ✓ | BbhDyckLanguagesBj65correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 003 |
| ✓ | BbhDyckLanguagesBj66correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 028 OUT5 493 |
| ✓ | BbhDyckLanguagesBj67correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT2 474 |
| ✓ | BbhDyckLanguagesBj68correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT385 |
| ✓ | BbhDyckLanguagesBj69correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT1 243 |
| ✓ | BbhDyckLanguagesBj70correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 004 OUT2 714 |
| ✓ | BbhDyckLanguagesBj71correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 029 OUT9 930 |
| ✓ | BbhDyckLanguagesBj72correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 421 |
| ✓ | BbhDyckLanguagesBj73correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 037 OUT10 319 |
| ✓ | BbhDyckLanguagesBj74correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 072 OUT3 196 |
| ✓ | BbhDyckLanguagesBj75correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 030 OUT4 081 |
| ✓ | BbhDyckLanguagesBj76correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT1 409 |
| ✓ | BbhDyckLanguagesBj77correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT945 |
| ✓ | BbhDyckLanguagesBj78correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 022 OUT1 908 |
| ✓ | BbhDyckLanguagesBj79correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 008 OUT2 000 |
| ✓ | BbhDyckLanguagesBj80correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT1 601 |
| ✓ | BbhDyckLanguagesBj81correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT1 730 |
| ✓ | BbhDyckLanguagesBj82correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT398 |
Reasoning - BBH Dyck languages - close the brackets (set 3)
28 passed4 failed |
87.5% |
IN31 966 OUT70 765 | |
| ✓ | BbhDyckLanguagesBj100correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT510 |
| ✓ | BbhDyckLanguagesBj101correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT893 |
| × | BbhDyckLanguagesBj102correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> >")
Context: BbhDyckLanguagesBj102: expected > >, got [>>>] |
Failed | IN1 047 OUT2 386 |
| ✓ | BbhDyckLanguagesBj103correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT370 |
| ✓ | BbhDyckLanguagesBj104correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT576 |
| ✓ | BbhDyckLanguagesBj105correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT413 |
| × | BbhDyckLanguagesBj106correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("]")
Context: BbhDyckLanguagesBj106: expected ], got [) ]] |
Failed | IN1 010 OUT3 228 |
| ✓ | BbhDyckLanguagesBj107correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT439 |
| × | BbhDyckLanguagesBj108correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> } >")
Context: BbhDyckLanguagesBj108: expected > } >, got [< { < ( ) > } >] |
Failed | IN978 OUT781 |
| ✓ | BbhDyckLanguagesBj109correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 066 OUT5 855 |
| ✓ | BbhDyckLanguagesBj110correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT634 |
| ✓ | BbhDyckLanguagesBj111correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 012 OUT4 882 |
| ✓ | BbhDyckLanguagesBj112correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 065 OUT7 909 |
| ✓ | BbhDyckLanguagesBj113correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT333 |
| ✓ | BbhDyckLanguagesBj114correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT800 |
| ✓ | BbhDyckLanguagesBj83correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT368 |
| ✓ | BbhDyckLanguagesBj84correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT576 |
| ✓ | BbhDyckLanguagesBj85correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT603 |
| ✓ | BbhDyckLanguagesBj86correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT352 |
| ✓ | BbhDyckLanguagesBj87correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 031 OUT3 364 |
| ✓ | BbhDyckLanguagesBj88correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 059 OUT6 543 |
| ✓ | BbhDyckLanguagesBj89correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT1 531 |
| ✓ | BbhDyckLanguagesBj90correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT2 706 |
| ✓ | BbhDyckLanguagesBj91correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT980 |
| ✓ | BbhDyckLanguagesBj92correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT710 |
| ✓ | BbhDyckLanguagesBj93correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT665 |
| ✓ | BbhDyckLanguagesBj94correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 017 OUT7 792 |
| × | BbhDyckLanguagesBj95correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") )")
Context: BbhDyckLanguagesBj95: expected ) ), got [)))] |
Failed | IN1 059 OUT11 623 |
| ✓ | BbhDyckLanguagesBj96correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT493 |
| ✓ | BbhDyckLanguagesBj97correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT321 |
| ✓ | BbhDyckLanguagesBj98correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT409 |
| ✓ | BbhDyckLanguagesBj99correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 720 |
Reasoning - BBH Dyck languages - close the brackets (set 4)
25 passed7 failed |
78.1% |
IN31 909 OUT57 207 | |
| ✓ | BbhDyckLanguagesBj115correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT914 |
| ✓ | BbhDyckLanguagesBj116correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 005 OUT1 120 |
| ✓ | BbhDyckLanguagesBj117correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 007 OUT1 263 |
| ✓ | BbhDyckLanguagesBj118correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT1 005 |
| ✓ | BbhDyckLanguagesBj119correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT489 |
| ✓ | BbhDyckLanguagesBj120correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 019 OUT3 283 |
| × | BbhDyckLanguagesBj121correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] }")
Context: BbhDyckLanguagesBj121: expected ] }, got [{ [ < [ ] > ] } { ( ) [ < > ] [ [ [ ] ] ] }] |
Failed | IN993 OUT2 775 |
| ✓ | BbhDyckLanguagesBj122correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 042 OUT8 456 |
| ✓ | BbhDyckLanguagesBj123correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT568 |
| × | BbhDyckLanguagesBj124correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> > }")
Context: BbhDyckLanguagesBj124: expected > > }, got [>}] |
Failed | IN1 004 OUT987 |
| ✓ | BbhDyckLanguagesBj125correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 660 |
| ✓ | BbhDyckLanguagesBj126correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT765 |
| ✓ | BbhDyckLanguagesBj127correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 017 OUT1 609 |
| × | BbhDyckLanguagesBj128correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(">")
Context: BbhDyckLanguagesBj128: expected >, got [< ( )>] |
Failed | IN976 OUT568 |
| ✓ | BbhDyckLanguagesBj129correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT289 |
| ✓ | BbhDyckLanguagesBj130correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT665 |
| ✓ | BbhDyckLanguagesBj131correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 009 OUT1 109 |
| × | BbhDyckLanguagesBj132correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") >")
Context: BbhDyckLanguagesBj132: expected ) >, got [< < > ( [ ]) >] |
Failed | IN979 OUT695 |
| × | BbhDyckLanguagesBj133correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} >")
Context: BbhDyckLanguagesBj133: expected } >, got []}>] |
Failed | IN987 OUT788 |
| ✓ | BbhDyckLanguagesBj134correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT785 |
| ✓ | BbhDyckLanguagesBj135correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 024 OUT5 972 |
| ✓ | BbhDyckLanguagesBj136correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT614 |
| ✓ | BbhDyckLanguagesBj137correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT595 |
| ✓ | BbhDyckLanguagesBj138correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT1 595 |
| ✓ | BbhDyckLanguagesBj139correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 050 OUT9 833 |
| × | BbhDyckLanguagesBj140correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ) )")
Context: BbhDyckLanguagesBj140: expected > ) ), got [>)] |
Failed | IN1 010 OUT1 518 |
| × | BbhDyckLanguagesBj141correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") )")
Context: BbhDyckLanguagesBj141: expected ) ), got [( ) ( ( ) )] |
Failed | IN977 OUT351 |
| ✓ | BbhDyckLanguagesBj142correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT570 |
| ✓ | BbhDyckLanguagesBj143correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT349 |
| ✓ | BbhDyckLanguagesBj144correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT974 |
| ✓ | BbhDyckLanguagesBj145correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 048 OUT4 418 |
| ✓ | BbhDyckLanguagesBj146correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT625 |
Reasoning - BBH Dyck languages - close the brackets (set 5)
26 passed6 failed |
81.3% |
IN31 812 OUT63 954 | |
| × | BbhDyckLanguagesBj147correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(">")
Context: BbhDyckLanguagesBj147: expected >, got [>>] |
Failed | IN976 OUT180 |
| ✓ | BbhDyckLanguagesBj148correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 018 OUT5 776 |
| ✓ | BbhDyckLanguagesBj149correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT802 |
| ✓ | BbhDyckLanguagesBj150correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT543 |
| ✓ | BbhDyckLanguagesBj151correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT570 |
| ✓ | BbhDyckLanguagesBj152correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT782 |
| ✓ | BbhDyckLanguagesBj153correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 009 OUT2 016 |
| ✓ | BbhDyckLanguagesBj154correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 008 OUT1 620 |
| × | BbhDyckLanguagesBj155correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> >")
Context: BbhDyckLanguagesBj155: expected > >, got [>] |
Failed | IN1 011 OUT7 867 |
| ✓ | BbhDyckLanguagesBj156correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT1 009 |
| ✓ | BbhDyckLanguagesBj157correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT611 |
| ✓ | BbhDyckLanguagesBj158correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT573 |
| ✓ | BbhDyckLanguagesBj159correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT786 |
| ✓ | BbhDyckLanguagesBj160correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 020 OUT1 531 |
| × | BbhDyckLanguagesBj161correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ) }")
Context: BbhDyckLanguagesBj161: expected ) ) }, got [{(())}] |
Failed | IN976 OUT679 |
| ✓ | BbhDyckLanguagesBj162correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 079 |
| ✓ | BbhDyckLanguagesBj163correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT1 174 |
| ✓ | BbhDyckLanguagesBj164correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT2 149 |
| ✓ | BbhDyckLanguagesBj165correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT822 |
| ✓ | BbhDyckLanguagesBj166correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT645 |
| ✓ | BbhDyckLanguagesBj167correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 008 OUT3 872 |
| ✓ | BbhDyckLanguagesBj168correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT123 |
| ✓ | BbhDyckLanguagesBj169correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT772 |
| ✓ | BbhDyckLanguagesBj170correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 023 OUT5 122 |
| × | BbhDyckLanguagesBj171correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> } >")
Context: BbhDyckLanguagesBj171: expected > } >, got [}>] |
Failed | IN1 002 OUT4 770 |
| ✓ | BbhDyckLanguagesBj172correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT570 |
| × | BbhDyckLanguagesBj173correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> > >")
Context: BbhDyckLanguagesBj173: expected > > >, got [< < [ ( < [ ( { [ ( ) ] [ ( ( { [ ] } ) ) ] } ) ] > ) ] < >>>] |
Failed | IN1 002 OUT1 334 |
| ✓ | BbhDyckLanguagesBj174correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 064 OUT11 630 |
| ✓ | BbhDyckLanguagesBj175correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 004 OUT1 762 |
| × | BbhDyckLanguagesBj176correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} ) >")
Context: BbhDyckLanguagesBj176: expected } ) >, got [}>] |
Failed | IN982 OUT1 220 |
| ✓ | BbhDyckLanguagesBj177correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT361 |
| ✓ | BbhDyckLanguagesBj178correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 204 |
Reasoning - BBH Dyck languages - close the brackets (set 6)
23 passed7 failed |
76.7% |
IN29 940 OUT74 718 | |
| ✓ | BbhDyckLanguagesBj179correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT461 |
| ✓ | BbhDyckLanguagesBj180correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT698 |
| ✓ | BbhDyckLanguagesBj181correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT1 004 |
| ✓ | BbhDyckLanguagesBj182correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT490 |
| ✓ | BbhDyckLanguagesBj183correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT719 |
| × | BbhDyckLanguagesBj184correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(">")
Context: BbhDyckLanguagesBj184: expected >, got [`>)))]>)>]>>}]>}>>]>>]>>]>>]`
Let's format cleanly: `>)))]>)>]>>}]>}>>]>>]>>]>>]`] |
Failed | IN1 068 OUT12 747 |
| ✓ | BbhDyckLanguagesBj185correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT567 |
| × | BbhDyckLanguagesBj186correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ] ]")
Context: BbhDyckLanguagesBj186: expected ) ] ], got []]] |
Failed | IN1 036 OUT7 648 |
| ✓ | BbhDyckLanguagesBj187correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT599 |
| ✓ | BbhDyckLanguagesBj188correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT995 |
| × | BbhDyckLanguagesBj189correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ]")
Context: BbhDyckLanguagesBj189: expected ) ], got [)] |
Failed | IN1 051 OUT6 457 |
| ✓ | BbhDyckLanguagesBj190correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT585 |
| ✓ | BbhDyckLanguagesBj191correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT1 538 |
| ✓ | BbhDyckLanguagesBj192correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT745 |
| ✓ | BbhDyckLanguagesBj193correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT353 |
| ✓ | BbhDyckLanguagesBj194correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 004 OUT1 019 |
| × | BbhDyckLanguagesBj195correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] >")
Context: BbhDyckLanguagesBj195: expected ] >, got [}]]>) )] |
Failed | IN1 037 OUT11 506 |
| × | BbhDyckLanguagesBj196correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} ) >")
Context: BbhDyckLanguagesBj196: expected } ) >, got [< ( { } ) >] |
Failed | IN976 OUT247 |
| ✓ | BbhDyckLanguagesBj197correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT670 |
| ✓ | BbhDyckLanguagesBj198correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 046 OUT5 658 |
| ✓ | BbhDyckLanguagesBj199correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 021 OUT1 764 |
| × | BbhDyckLanguagesBj200correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBj200: expected ), got [} ] }] |
Failed | IN1 066 OUT9 633 |
| ✓ | BbhDyckLanguagesBj201correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT1 402 |
| ✓ | BbhDyckLanguagesBj202correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT732 |
| ✓ | BbhDyckLanguagesBj203correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT358 |
| ✓ | BbhDyckLanguagesBj204correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT1 457 |
| ✓ | BbhDyckLanguagesBj205correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT955 |
| ✓ | BbhDyckLanguagesBj206correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT996 |
| × | BbhDyckLanguagesBj207correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] > }")
Context: BbhDyckLanguagesBj207: expected ] > }, got []}>] |
Failed | IN992 OUT1 158 |
| ✓ | BbhDyckLanguagesBj208correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT1 557 |
Reasoning - BBH Dyck languages - close the brackets (set 7)
19 passed13 failed |
59.4% |
IN32 026 OUT67 158 | |
| × | BbhDyckLanguagesBk209correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} } )")
Context: BbhDyckLanguagesBk209: expected } } ), got [>>}}}}}}}] |
Failed | IN1 028 OUT10 392 |
| ✓ | BbhDyckLanguagesBk210correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT609 |
| × | BbhDyckLanguagesBk211correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] } )")
Context: BbhDyckLanguagesBk211: expected ] } ), got [] }] |
Failed | IN978 OUT505 |
| × | BbhDyckLanguagesBk212correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} )")
Context: BbhDyckLanguagesBk212: expected } ), got [}] |
Failed | IN1 069 OUT6 438 |
| × | BbhDyckLanguagesBk213correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") >")
Context: BbhDyckLanguagesBk213: expected ) >, got [>] |
Failed | IN989 OUT891 |
| × | BbhDyckLanguagesBk214correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} ] }")
Context: BbhDyckLanguagesBk214: expected } ] }, got [}]}}] |
Failed | IN982 OUT1 017 |
| × | BbhDyckLanguagesBk215correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") > }")
Context: BbhDyckLanguagesBk215: expected ) > }, got [>}] |
Failed | IN1 030 OUT4 286 |
| ✓ | BbhDyckLanguagesBk216correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT620 |
| ✓ | BbhDyckLanguagesBk217correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT453 |
| × | BbhDyckLanguagesBk218correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBk218: expected ), got [] |
Failed | IN976 OUT287 |
| ✓ | BbhDyckLanguagesBk219correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT1 117 |
| ✓ | BbhDyckLanguagesBk220correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT410 |
| × | BbhDyckLanguagesBk221correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") ] ]")
Context: BbhDyckLanguagesBk221: expected ) ] ], got []]] |
Failed | IN1 004 OUT1 143 |
| ✓ | BbhDyckLanguagesBk222correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 003 |
| ✓ | BbhDyckLanguagesBk223correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT343 |
| × | BbhDyckLanguagesBk224correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ]")
Context: BbhDyckLanguagesBk224: expected > ], got [[ <> ]] |
Failed | IN977 OUT1 188 |
| ✓ | BbhDyckLanguagesBk225correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 066 OUT3 931 |
| × | BbhDyckLanguagesBk226correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ]")
Context: BbhDyckLanguagesBk226: expected > ], got [] >] |
Failed | IN1 009 OUT1 262 |
| × | BbhDyckLanguagesBk227correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} } )")
Context: BbhDyckLanguagesBk227: expected } } ), got [}}] |
Failed | IN1 070 OUT5 440 |
| × | BbhDyckLanguagesBk228correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") >")
Context: BbhDyckLanguagesBk228: expected ) >, got [>] |
Failed | IN975 OUT589 |
| ✓ | BbhDyckLanguagesBk229correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 041 |
| ✓ | BbhDyckLanguagesBk230correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 306 |
| ✓ | BbhDyckLanguagesBk231correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT725 |
| ✓ | BbhDyckLanguagesBk232correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT1 580 |
| ✓ | BbhDyckLanguagesBk233correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT2 012 |
| ✓ | BbhDyckLanguagesBk234correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT991 |
| ✓ | BbhDyckLanguagesBk235correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 034 OUT2 815 |
| × | BbhDyckLanguagesBk236correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") } >")
Context: BbhDyckLanguagesBk236: expected ) } >, got [}>] |
Failed | IN990 OUT1 124 |
| ✓ | BbhDyckLanguagesBk237correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 070 OUT10 542 |
| ✓ | BbhDyckLanguagesBk238correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT503 |
| ✓ | BbhDyckLanguagesBk239correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT713 |
| ✓ | BbhDyckLanguagesBk240correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 882 |
Reasoning - BBH formal fallacies (set 1)
26 passed6 failed |
81.3% |
IN33 806 OUT30 284 | |
| × | BbhDyckLanguagesBk241correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(")")
Context: BbhDyckLanguagesBk241: expected ), got [[ ] [ { } ]] |
Failed | IN982 OUT752 |
| × | BbhDyckLanguagesBk242correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets(") }")
Context: BbhDyckLanguagesBk242: expected ) }, got [}] |
Failed | IN983 OUT604 |
| × | BbhDyckLanguagesBk243correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} )")
Context: BbhDyckLanguagesBk243: expected } ), got [}] |
Failed | IN999 OUT1 479 |
| × | BbhDyckLanguagesBk244correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ) }")
Context: BbhDyckLanguagesBk244: expected > ) }, got [> }] |
Failed | IN976 OUT478 |
| × | BbhDyckLanguagesBk245correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("} ) }")
Context: BbhDyckLanguagesBk245: expected } ) }, got [}}] |
Failed | IN998 OUT1 577 |
| ✓ | BbhDyckLanguagesBk246correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 019 OUT1 832 |
| × | BbhDyckLanguagesBk247correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("> ) )")
Context: BbhDyckLanguagesBk247: expected > ) ), got [>] |
Failed | IN992 OUT964 |
| ✓ | BbhDyckLanguagesBk248correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT445 |
| ✓ | BbhDyckLanguagesBk249correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT863 |
| ✓ | BbhFormalFallaciesBk10decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT673 |
| ✓ | BbhFormalFallaciesBk11decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT788 |
| ✓ | BbhFormalFallaciesBk12decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT582 |
| ✓ | BbhFormalFallaciesBk13decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT839 |
| ✓ | BbhFormalFallaciesBk14decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 271 |
| ✓ | BbhFormalFallaciesBk15decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT1 034 |
| ✓ | BbhFormalFallaciesBk16decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT975 |
| ✓ | BbhFormalFallaciesBk17decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT508 |
| ✓ | BbhFormalFallaciesBk18decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT1 180 |
| ✓ | BbhFormalFallaciesBk19decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT809 |
| ✓ | BbhFormalFallaciesBk20decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT1 164 |
| ✓ | BbhFormalFallaciesBk21decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT717 |
| ✓ | BbhFormalFallaciesBk22decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT859 |
| ✓ | BbhFormalFallaciesBk23decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 178 |
| ✓ | BbhFormalFallaciesBk24decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 728 |
| ✓ | BbhFormalFallaciesBk25decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT696 |
| ✓ | BbhFormalFallaciesBk26decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT390 |
| ✓ | BbhFormalFallaciesBk27decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT703 |
| ✓ | BbhFormalFallaciesBk28decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT852 |
| ✓ | BbhFormalFallaciesBk29decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT1 274 |
| ✓ | BbhFormalFallaciesBk30decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 929 |
| ✓ | BbhFormalFallaciesBk8decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT506 |
| ✓ | BbhFormalFallaciesBk9decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT635 |
Reasoning - BBH formal fallacies (set 2)
32 passed |
100.0% |
IN34 415 OUT30 759 | |
| ✓ | BbhFormalFallaciesBk31decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT988 |
| ✓ | BbhFormalFallaciesBk32decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT1 139 |
| ✓ | BbhFormalFallaciesBk33decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT699 |
| ✓ | BbhFormalFallaciesBk34decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 408 |
| ✓ | BbhFormalFallaciesBk35decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT751 |
| ✓ | BbhFormalFallaciesBk36decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 193 |
| ✓ | BbhFormalFallaciesBk37decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 766 |
| ✓ | BbhFormalFallaciesBk38decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT1 024 |
| ✓ | BbhFormalFallaciesBk39decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 144 |
| ✓ | BbhFormalFallaciesBk40decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 258 |
| ✓ | BbhFormalFallaciesBk41decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT414 |
| ✓ | BbhFormalFallaciesBk42decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT724 |
| ✓ | BbhFormalFallaciesBk43decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT1 303 |
| ✓ | BbhFormalFallaciesBk44decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 517 |
| ✓ | BbhFormalFallaciesBk45decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT559 |
| ✓ | BbhFormalFallaciesBk46decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT946 |
| ✓ | BbhFormalFallaciesBk47decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT989 |
| ✓ | BbhFormalFallaciesBk48decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT539 |
| ✓ | BbhFormalFallaciesBk49decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT1 422 |
| ✓ | BbhFormalFallaciesBk50decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT900 |
| ✓ | BbhFormalFallaciesBk51decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT1 061 |
| ✓ | BbhFormalFallaciesBk52decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 034 OUT561 |
| ✓ | BbhFormalFallaciesBk53decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT820 |
| ✓ | BbhFormalFallaciesBk54decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT607 |
| ✓ | BbhFormalFallaciesBk55decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT556 |
| ✓ | BbhFormalFallaciesBk56decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT511 |
| ✓ | BbhFormalFallaciesBk57decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT495 |
| ✓ | BbhFormalFallaciesBk58decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT1 883 |
| ✓ | BbhFormalFallaciesBk59decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 055 |
| ✓ | BbhFormalFallaciesBk60decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT1 090 |
| ✓ | BbhFormalFallaciesBk61decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT807 |
| ✓ | BbhFormalFallaciesBk62decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT630 |
Reasoning - BBH formal fallacies (set 3)
32 passed |
100.0% |
IN34 624 OUT35 688 | |
| ✓ | BbhFormalFallaciesBk63decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT1 174 |
| ✓ | BbhFormalFallaciesBk64decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT832 |
| ✓ | BbhFormalFallaciesBk65decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT1 225 |
| ✓ | BbhFormalFallaciesBk66decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT602 |
| ✓ | BbhFormalFallaciesBk67decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT870 |
| ✓ | BbhFormalFallaciesBk68decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT2 517 |
| ✓ | BbhFormalFallaciesBk69decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 120 |
| ✓ | BbhFormalFallaciesBk70decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT967 |
| ✓ | BbhFormalFallaciesBk71decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT680 |
| ✓ | BbhFormalFallaciesBk72decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT708 |
| ✓ | BbhFormalFallaciesBk73decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT655 |
| ✓ | BbhFormalFallaciesBk74decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT975 |
| ✓ | BbhFormalFallaciesBk75decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 554 |
| ✓ | BbhFormalFallaciesBk76decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 974 |
| ✓ | BbhFormalFallaciesBk77decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT1 674 |
| ✓ | BbhFormalFallaciesBk78decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 299 |
| ✓ | BbhFormalFallaciesBk79decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT639 |
| ✓ | BbhFormalFallaciesBk80decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT1 050 |
| ✓ | BbhFormalFallaciesBk81decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT2 407 |
| ✓ | BbhFormalFallaciesBk82decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT573 |
| ✓ | BbhFormalFallaciesBk83decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT1 222 |
| ✓ | BbhFormalFallaciesBk84decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT709 |
| ✓ | BbhFormalFallaciesBk85decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 078 |
| ✓ | BbhFormalFallaciesBk86decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT1 347 |
| ✓ | BbhFormalFallaciesBk87decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 253 |
| ✓ | BbhFormalFallaciesBk88decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 010 |
| ✓ | BbhFormalFallaciesBk89decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT524 |
| ✓ | BbhFormalFallaciesBk90decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT1 570 |
| ✓ | BbhFormalFallaciesBk91decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT1 421 |
| ✓ | BbhFormalFallaciesBk92decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT714 |
| ✓ | BbhFormalFallaciesBk93decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT656 |
| ✓ | BbhFormalFallaciesBk94decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT689 |
Reasoning - BBH formal fallacies (set 4)
30 passed |
100.0% |
IN32 188 OUT28 966 | |
| ✓ | BbhFormalFallaciesBk100decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT1 176 |
| ✓ | BbhFormalFallaciesBk101decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT1 397 |
| ✓ | BbhFormalFallaciesBk102decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT525 |
| ✓ | BbhFormalFallaciesBk103decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT634 |
| ✓ | BbhFormalFallaciesBk104decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT1 025 |
| ✓ | BbhFormalFallaciesBk105decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 164 |
| ✓ | BbhFormalFallaciesBk106decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT508 |
| ✓ | BbhFormalFallaciesBk107decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT764 |
| ✓ | BbhFormalFallaciesBk108decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT900 |
| ✓ | BbhFormalFallaciesBk109decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT554 |
| ✓ | BbhFormalFallaciesBk110decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT1 115 |
| ✓ | BbhFormalFallaciesBk111decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 558 |
| ✓ | BbhFormalFallaciesBk112decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT467 |
| ✓ | BbhFormalFallaciesBk113decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT708 |
| ✓ | BbhFormalFallaciesBk114decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT710 |
| ✓ | BbhFormalFallaciesBk115decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT807 |
| ✓ | BbhFormalFallaciesBk116decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT601 |
| ✓ | BbhFormalFallaciesBk117decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT383 |
| ✓ | BbhFormalFallaciesBk118decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT505 |
| ✓ | BbhFormalFallaciesBk119decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT752 |
| ✓ | BbhFormalFallaciesBk120decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT483 |
| ✓ | BbhFormalFallaciesBk121decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT607 |
| ✓ | BbhFormalFallaciesBk122decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT872 |
| ✓ | BbhFormalFallaciesBk123decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT895 |
| ✓ | BbhFormalFallaciesBk124decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT784 |
| ✓ | BbhFormalFallaciesBk95decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 509 |
| ✓ | BbhFormalFallaciesBk96decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT884 |
| ✓ | BbhFormalFallaciesBk97decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 138 OUT3 802 |
| ✓ | BbhFormalFallaciesBk98decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT997 |
| ✓ | BbhFormalFallaciesBk99decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 880 |
Reasoning - BBH formal fallacies (set 5)
32 passed |
100.0% |
IN34 494 OUT31 170 | |
| ✓ | BbhFormalFallacies125BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT795 |
| ✓ | BbhFormalFallacies126BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT765 |
| ✓ | BbhFormalFallacies127BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT559 |
| ✓ | BbhFormalFallacies128BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT721 |
| ✓ | BbhFormalFallacies129BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT630 |
| ✓ | BbhFormalFallacies130BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT537 |
| ✓ | BbhFormalFallacies131BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT1 128 |
| ✓ | BbhFormalFallacies132BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT755 |
| ✓ | BbhFormalFallacies133BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT447 |
| ✓ | BbhFormalFallacies134BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT577 |
| ✓ | BbhFormalFallacies135BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 060 |
| ✓ | BbhFormalFallacies136BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT1 519 |
| ✓ | BbhFormalFallacies137BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT807 |
| ✓ | BbhFormalFallacies138BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 060 |
| ✓ | BbhFormalFallacies139BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT962 |
| ✓ | BbhFormalFallacies140BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT901 |
| ✓ | BbhFormalFallacies141BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 087 |
| ✓ | BbhFormalFallacies142BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT651 |
| ✓ | BbhFormalFallacies143BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 798 |
| ✓ | BbhFormalFallacies144BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT1 157 |
| ✓ | BbhFormalFallacies145BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT869 |
| ✓ | BbhFormalFallacies146BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT661 |
| ✓ | BbhFormalFallacies147BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT660 |
| ✓ | BbhFormalFallacies148BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT2 186 |
| ✓ | BbhFormalFallacies149BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 438 |
| ✓ | BbhFormalFallacies150BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT1 288 |
| ✓ | BbhFormalFallacies151BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT773 |
| ✓ | BbhFormalFallacies152BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT821 |
| ✓ | BbhFormalFallacies153BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT1 471 |
| ✓ | BbhFormalFallacies154BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT758 |
| ✓ | BbhFormalFallacies155BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 071 |
| ✓ | BbhFormalFallacies156BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 258 |
Reasoning - BBH formal fallacies (set 6)
32 passed |
100.0% |
IN34 489 OUT32 534 | |
| ✓ | BbhFormalFallacies157BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT831 |
| ✓ | BbhFormalFallacies158BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT993 |
| ✓ | BbhFormalFallacies159BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT1 238 |
| ✓ | BbhFormalFallacies160BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT1 012 |
| ✓ | BbhFormalFallacies161BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT786 |
| ✓ | BbhFormalFallacies162BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT839 |
| ✓ | BbhFormalFallacies163BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT761 |
| ✓ | BbhFormalFallacies164BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 565 |
| ✓ | BbhFormalFallacies165BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT1 468 |
| ✓ | BbhFormalFallacies166BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT865 |
| ✓ | BbhFormalFallacies167BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT549 |
| ✓ | BbhFormalFallacies168BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT848 |
| ✓ | BbhFormalFallacies169BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 455 |
| ✓ | BbhFormalFallacies170BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT572 |
| ✓ | BbhFormalFallacies171BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT1 929 |
| ✓ | BbhFormalFallacies172BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT2 085 |
| ✓ | BbhFormalFallacies173BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT522 |
| ✓ | BbhFormalFallacies174BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT763 |
| ✓ | BbhFormalFallacies175BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT505 |
| ✓ | BbhFormalFallacies176BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT757 |
| ✓ | BbhFormalFallacies177BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT1 484 |
| ✓ | BbhFormalFallacies178BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 826 |
| ✓ | BbhFormalFallacies179BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 157 |
| ✓ | BbhFormalFallacies180BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT475 |
| ✓ | BbhFormalFallacies181BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 025 |
| ✓ | BbhFormalFallacies182BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 253 |
| ✓ | BbhFormalFallacies183BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT756 |
| ✓ | BbhFormalFallacies184BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT586 |
| ✓ | BbhFormalFallacies185BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 038 OUT484 |
| ✓ | BbhFormalFallacies186BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT525 |
| ✓ | BbhFormalFallacies187BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT1 283 |
| ✓ | BbhFormalFallacies188BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 337 |
Reasoning - BBH formal fallacies (set 7)
32 passed |
100.0% |
IN34 363 OUT31 270 | |
| ✓ | BbhFormalFallacies189BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT467 |
| ✓ | BbhFormalFallacies190BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT941 |
| ✓ | BbhFormalFallacies191BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT518 |
| ✓ | BbhFormalFallacies192BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT1 014 |
| ✓ | BbhFormalFallacies193BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT516 |
| ✓ | BbhFormalFallacies194BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT356 |
| ✓ | BbhFormalFallacies195BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT689 |
| ✓ | BbhFormalFallacies196BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 584 |
| ✓ | BbhFormalFallacies197BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT517 |
| ✓ | BbhFormalFallacies198BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT1 095 |
| ✓ | BbhFormalFallacies199BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 412 |
| ✓ | BbhFormalFallacies200BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT926 |
| ✓ | BbhFormalFallacies201BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT1 385 |
| ✓ | BbhFormalFallacies202BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT1 788 |
| ✓ | BbhFormalFallacies203BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT964 |
| ✓ | BbhFormalFallacies204BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT313 |
| ✓ | BbhFormalFallacies205BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT984 |
| ✓ | BbhFormalFallacies206BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT2 235 |
| ✓ | BbhFormalFallacies207BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT843 |
| ✓ | BbhFormalFallacies208BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT965 |
| ✓ | BbhFormalFallacies209BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 012 |
| ✓ | BbhFormalFallacies210BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT659 |
| ✓ | BbhFormalFallacies211BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT1 436 |
| ✓ | BbhFormalFallacies212BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT631 |
| ✓ | BbhFormalFallacies213BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT1 552 |
| ✓ | BbhFormalFallacies214BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT829 |
| ✓ | BbhFormalFallacies215BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT606 |
| ✓ | BbhFormalFallacies216BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 504 |
| ✓ | BbhFormalFallacies217BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT1 241 |
| ✓ | BbhFormalFallacies218BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT902 |
| ✓ | BbhFormalFallacies219BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT748 |
| ✓ | BbhFormalFallacies220BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT638 |
Reasoning - BBH formal fallacies (set 8)
28 passed1 failed |
96.6% |
IN31 240 OUT30 497 | |
| ✓ | BbhFormalFallacies221BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT603 |
| ✓ | BbhFormalFallacies222BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT886 |
| ✓ | BbhFormalFallacies223BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 121 OUT1 403 |
| ✓ | BbhFormalFallacies224BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT1 131 |
| ✓ | BbhFormalFallacies225BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT438 |
| ✓ | BbhFormalFallacies226BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT1 629 |
| ✓ | BbhFormalFallacies227BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 427 |
| ✓ | BbhFormalFallacies228BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 613 |
| ✓ | BbhFormalFallacies229BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 097 |
| ✓ | BbhFormalFallacies230BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT1 621 |
| ✓ | BbhFormalFallacies231BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT993 |
| ✓ | BbhFormalFallacies232BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT656 |
| ✓ | BbhFormalFallacies233BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 033 |
| ✓ | BbhFormalFallacies234BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT1 807 |
| ✓ | BbhFormalFallacies235BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT443 |
| ✓ | BbhFormalFallacies236BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT942 |
| ✓ | BbhFormalFallacies237BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT786 |
| ✓ | BbhFormalFallacies238BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT756 |
| ✓ | BbhFormalFallacies239BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 492 |
| ✓ | BbhFormalFallacies240BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 197 |
| ✓ | BbhFormalFallacies241BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT726 |
| ✓ | BbhFormalFallacies242BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 483 |
| ✓ | BbhFormalFallacies243BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 801 |
| ✓ | BbhFormalFallacies244BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT961 |
| ✓ | BbhFormalFallacies245BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT657 |
| × | BbhFormalFallacies246BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("invalid")
Context: BbhFormalFallacies246BL: expected invalid, got [valid] |
Failed | IN1 108 OUT1 135 |
| ✓ | BbhFormalFallacies247BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT469 |
| ✓ | BbhFormalFallacies248BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT606 |
| ✓ | BbhFormalFallacies249BLdecide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT706 |
Reasoning - BBH geometric shapes (SVG paths) (set 1)
24 passed9 failed |
72.7% |
IN36 670 OUT48 321 | |
| ✓ | BbhGeometricShapes10BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 206 |
| ✓ | BbhGeometricShapes11BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT896 |
| × | BbhGeometricShapes12BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes12BL: expected (K), got [(A)] |
Failed | IN1 103 OUT1 602 |
| × | BbhGeometricShapes13BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes13BL: expected (K), got [(A)] |
Failed | IN1 099 OUT1 042 |
| × | BbhGeometricShapes14BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes14BL: expected (K), got [(H)] |
Failed | IN1 096 OUT2 937 |
| ✓ | BbhGeometricShapes15BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 922 |
| ✓ | BbhGeometricShapes16BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 130 |
| ✓ | BbhGeometricShapes17BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 196 OUT1 401 |
| × | BbhGeometricShapes18BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes18BL: expected (K), got [(H)] |
Failed | IN1 112 OUT2 835 |
| ✓ | BbhGeometricShapes19BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 646 |
| ✓ | BbhGeometricShapes20BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT948 |
| ✓ | BbhGeometricShapes21BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT1 291 |
| ✓ | BbhGeometricShapes22BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 369 |
| ✓ | BbhGeometricShapes23BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 166 OUT1 286 |
| ✓ | BbhGeometricShapes24BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT1 282 |
| ✓ | BbhGeometricShapes25BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT2 145 |
| ✓ | BbhGeometricShapes26BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT491 |
| ✓ | BbhGeometricShapes27BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 002 |
| ✓ | BbhGeometricShapes28BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT960 |
| × | BbhGeometricShapes29BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes29BL: expected (K), got [(H)] |
Failed | IN1 084 OUT908 |
| ✓ | BbhGeometricShapes30BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT867 |
| ✓ | BbhGeometricShapes31BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 299 |
| ✓ | BbhGeometricShapes32BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT3 871 |
| × | BbhGeometricShapes33BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes33BL: expected (K), got [(A)] |
Failed | IN1 103 OUT1 256 |
| × | BbhGeometricShapes34BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhGeometricShapes34BL: expected (C), got [(F)] |
Failed | IN1 117 OUT2 891 |
| ✓ | BbhGeometricShapes35BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 671 |
| ✓ | BbhGeometricShapes36BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 530 |
| ✓ | BbhGeometricShapes37BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT611 |
| × | BbhGeometricShapes38BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(I)")
Context: BbhGeometricShapes38BL: expected (I), got [J] |
Failed | IN1 089 OUT1 333 |
| ✓ | BbhGeometricShapes39BLThis SVG path element <path d="M 2.27,59.23 L 52.45,67.18"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT282 |
| ✓ | BbhGeometricShapes40BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT947 |
| × | BbhGeometricShapes8BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes8BL: expected (K), got [(H)] |
Failed | IN1 084 OUT2 300 |
| ✓ | BbhGeometricShapes9BLidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 164 |
Reasoning - BBH geometric shapes (SVG paths) (set 2)
25 passed7 failed |
78.1% |
IN35 097 OUT42 066 | |
| ✓ | BbhGeometricShapes41Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 607 |
| ✓ | BbhGeometricShapes42Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 334 |
| ✓ | BbhGeometricShapes43BmThis SVG path element <path d="M 98.96,92.35 L 44.27,28.27"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT327 |
| ✓ | BbhGeometricShapes44Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT593 |
| × | BbhGeometricShapes45Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes45Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT1 835 |
| × | BbhGeometricShapes46Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes46Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 103 OUT1 511 |
| ✓ | BbhGeometricShapes47Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 445 |
| ✓ | BbhGeometricShapes48Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT3 256 |
| ✓ | BbhGeometricShapes49Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 234 |
| ✓ | BbhGeometricShapes50Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 167 OUT1 832 |
| ✓ | BbhGeometricShapes51Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 725 |
| ✓ | BbhGeometricShapes52Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 084 |
| ✓ | BbhGeometricShapes53Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT860 |
| ✓ | BbhGeometricShapes54Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 861 |
| ✓ | BbhGeometricShapes55BmThis SVG path element <path d="M 32.95,20.92 L 31.65,35.07"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT305 |
| ✓ | BbhGeometricShapes56Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 657 |
| ✓ | BbhGeometricShapes57Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 370 |
| ✓ | BbhGeometricShapes58Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT2 802 |
| × | BbhGeometricShapes59Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes59Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 097 OUT1 103 |
| ✓ | BbhGeometricShapes60Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 929 |
| ✓ | BbhGeometricShapes61Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT742 |
| ✓ | BbhGeometricShapes62Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT1 274 |
| ✓ | BbhGeometricShapes63Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT944 |
| ✓ | BbhGeometricShapes64Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT528 |
| × | BbhGeometricShapes65Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes65Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 103 OUT1 526 |
| × | BbhGeometricShapes66Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes66Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 097 OUT2 376 |
| ✓ | BbhGeometricShapes67Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT889 |
| ✓ | BbhGeometricShapes68BmThis SVG path element <path d="M 11.79,16.93 L 56.17,80.16"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT266 |
| ✓ | BbhGeometricShapes69Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 271 |
| ✓ | BbhGeometricShapes70Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT631 |
| × | BbhGeometricShapes71Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes71Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 103 OUT1 192 |
| × | BbhGeometricShapes72Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes72Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 086 OUT757 |
Reasoning - BBH geometric shapes (SVG paths) (set 3)
26 passed6 failed |
81.3% |
IN35 038 OUT39 809 | |
| ✓ | BbhGeometricShapes100Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT929 |
| ✓ | BbhGeometricShapes101Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT540 |
| ✓ | BbhGeometricShapes102Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT1 047 |
| ✓ | BbhGeometricShapes103Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 798 |
| × | BbhGeometricShapes104Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes104Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 098 OUT1 725 |
| ✓ | BbhGeometricShapes73BmThis SVG path element <path d="M 31.97,19.98 L 38.55,43.43"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT302 |
| ✓ | BbhGeometricShapes74BmThis SVG path element <path d="M 81.33,89.86 L 65.20,41.90"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT335 |
| ✓ | BbhGeometricShapes75BmThis SVG path element <path d="M 27.49,66.87 L 19.65,56.41"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT335 |
| ✓ | BbhGeometricShapes76Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT1 345 |
| ✓ | BbhGeometricShapes77Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 764 |
| ✓ | BbhGeometricShapes78Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT2 051 |
| ✓ | BbhGeometricShapes79BmThis SVG path element <path d="M 14.31,45.36 L 17.22,37.11"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT367 |
| × | BbhGeometricShapes80Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes80Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 097 OUT2 810 |
| ✓ | BbhGeometricShapes81Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT2 469 |
| ✓ | BbhGeometricShapes82Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 385 |
| ✓ | BbhGeometricShapes83Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT844 |
| ✓ | BbhGeometricShapes84Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT822 |
| ✓ | BbhGeometricShapes85BmThis SVG path element <path d="M 0.58,63.64 L 34.42,42.20"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT331 |
| ✓ | BbhGeometricShapes86Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 746 |
| × | BbhGeometricShapes87Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes87Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 086 OUT2 174 |
| ✓ | BbhGeometricShapes88Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 195 |
| ✓ | BbhGeometricShapes89Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT861 |
| ✓ | BbhGeometricShapes90Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 400 |
| ✓ | BbhGeometricShapes91Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 087 |
| ✓ | BbhGeometricShapes92Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT614 |
| × | BbhGeometricShapes93Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes93Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 086 OUT3 683 |
| ✓ | BbhGeometricShapes94Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT968 |
| × | BbhGeometricShapes95Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes95Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 103 OUT901 |
| ✓ | BbhGeometricShapes96Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 138 |
| ✓ | BbhGeometricShapes97Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT713 |
| × | BbhGeometricShapes98Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes98Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 086 OUT1 199 |
| ✓ | BbhGeometricShapes99Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT931 |
Reasoning - BBH geometric shapes (SVG paths) (set 4)
24 passed8 failed |
75.0% |
IN35 330 OUT43 377 | |
| ✓ | BbhGeometricShapes105Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 166 OUT1 282 |
| ✓ | BbhGeometricShapes106Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 906 |
| ✓ | BbhGeometricShapes107BmThis SVG path element <path d="M 22.00,62.00 L 30.00,74.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT338 |
| ✓ | BbhGeometricShapes108Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 110 |
| × | BbhGeometricShapes109Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes109Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 103 OUT1 286 |
| × | BbhGeometricShapes110Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes110Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT1 329 |
| × | BbhGeometricShapes111Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes111Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 086 OUT912 |
| ✓ | BbhGeometricShapes112Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT2 724 |
| ✓ | BbhGeometricShapes113Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT1 287 |
| ✓ | BbhGeometricShapes114Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT1 652 |
| ✓ | BbhGeometricShapes115Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT714 |
| ✓ | BbhGeometricShapes116Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT543 |
| ✓ | BbhGeometricShapes117Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 183 OUT1 936 |
| × | BbhGeometricShapes118Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes118Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT1 615 |
| ✓ | BbhGeometricShapes119BmThis SVG path element <path d="M 17.11,37.49 L 43.48,13.53"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT209 |
| × | BbhGeometricShapes120Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes120Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 112 OUT2 526 |
| ✓ | BbhGeometricShapes121Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT5 181 |
| ✓ | BbhGeometricShapes122Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 656 |
| ✓ | BbhGeometricShapes123BmThis SVG path element <path d="M 51.00,18.00 L 5.00,62.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT373 |
| × | BbhGeometricShapes124Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes124Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT1 390 |
| ✓ | BbhGeometricShapes125Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 183 |
| ✓ | BbhGeometricShapes126Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 299 |
| ✓ | BbhGeometricShapes127Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 635 |
| × | BbhGeometricShapes128Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes128Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT1 167 |
| ✓ | BbhGeometricShapes129Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 164 OUT937 |
| ✓ | BbhGeometricShapes130Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 360 |
| ✓ | BbhGeometricShapes131Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 209 OUT999 |
| ✓ | BbhGeometricShapes132BmThis SVG path element <path d="M 26.00,23.00 L 19.00,16.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT345 |
| ✓ | BbhGeometricShapes133Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT726 |
| × | BbhGeometricShapes134Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes134Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 086 OUT2 206 |
| ✓ | BbhGeometricShapes135Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 156 |
| ✓ | BbhGeometricShapes136BmThis SVG path element <path d="M 46.92,78.61 L 46.94,45.83"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT395 |
Reasoning - BBH geometric shapes (SVG paths) (set 5)
26 passed5 failed |
83.9% |
IN34 220 OUT35 582 | |
| ✓ | BbhGeometricShapes137Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT840 |
| ✓ | BbhGeometricShapes138Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 195 |
| ✓ | BbhGeometricShapes139BmThis SVG path element <path d="M 58.33,52.18 L 34.28,31.10"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT344 |
| ✓ | BbhGeometricShapes140Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 365 |
| ✓ | BbhGeometricShapes141BmThis SVG path element <path d="M 38.35,49.41 L 31.18,9.15"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT272 |
| × | BbhGeometricShapes142Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes142Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT952 |
| ✓ | BbhGeometricShapes143Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT927 |
| ✓ | BbhGeometricShapes144Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 221 |
| × | BbhGeometricShapes145Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes145Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT1 492 |
| ✓ | BbhGeometricShapes146Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 213 |
| ✓ | BbhGeometricShapes147Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT2 291 |
| ✓ | BbhGeometricShapes148Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT676 |
| ✓ | BbhGeometricShapes149Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 099 |
| ✓ | BbhGeometricShapes150BmThis SVG path element <path d="M 90.41,82.38 L 45.53,97.84"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT258 |
| ✓ | BbhGeometricShapes151Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT1 289 |
| ✓ | BbhGeometricShapes152Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT856 |
| ✓ | BbhGeometricShapes153Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT648 |
| ✓ | BbhGeometricShapes154Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT560 |
| ✓ | BbhGeometricShapes155Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 462 |
| ✓ | BbhGeometricShapes156Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 278 |
| ✓ | BbhGeometricShapes157Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT3 116 |
| ✓ | BbhGeometricShapes158Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 643 |
| ✓ | BbhGeometricShapes159Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 483 |
| ✓ | BbhGeometricShapes160Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT466 |
| × | BbhGeometricShapes161Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes161Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 103 OUT1 327 |
| ✓ | BbhGeometricShapes162Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 183 OUT1 300 |
| × | BbhGeometricShapes163Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes163Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 125 OUT2 319 |
| × | BbhGeometricShapes164Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes164Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 099 OUT1 034 |
| ✓ | BbhGeometricShapes165Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT743 |
| ✓ | BbhGeometricShapes166Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT855 |
| ✓ | BbhGeometricShapes167Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT1 058 |
Reasoning - BBH geometric shapes (SVG paths) (set 6)
28 passed3 failed |
90.3% |
IN34 334 OUT45 232 | |
| ✓ | BbhGeometricShapes168Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT736 |
| ✓ | BbhGeometricShapes169Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 463 |
| ✓ | BbhGeometricShapes170Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 628 |
| ✓ | BbhGeometricShapes171Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 162 |
| ✓ | BbhGeometricShapes172Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 265 |
| ✓ | BbhGeometricShapes173Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 192 |
| ✓ | BbhGeometricShapes174Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 352 |
| ✓ | BbhGeometricShapes175Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT2 137 |
| ✓ | BbhGeometricShapes176Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 695 |
| ✓ | BbhGeometricShapes177Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT2 348 |
| ✓ | BbhGeometricShapes178Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT1 162 |
| ✓ | BbhGeometricShapes179Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT843 |
| ✓ | BbhGeometricShapes180Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 437 |
| × | BbhGeometricShapes181Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes181Bm: expected (K), got [a]
Values:
got = "a" |
Failed | IN1 101 OUT909 |
| × | BbhGeometricShapes182Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes182Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 086 OUT3 540 |
| ✓ | BbhGeometricShapes183Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 516 |
| ✓ | BbhGeometricShapes184BmThis SVG path element <path d="M 25.00,38.00 L 89.00,58.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT341 |
| ✓ | BbhGeometricShapes185Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 074 |
| ✓ | BbhGeometricShapes186Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT742 |
| ✓ | BbhGeometricShapes187BmThis SVG path element <path d="M 51.78,63.32 L 1.41,97.49"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT231 |
| ✓ | BbhGeometricShapes188Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT3 261 |
| ✓ | BbhGeometricShapes189Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT1 075 |
| ✓ | BbhGeometricShapes190Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT1 726 |
| ✓ | BbhGeometricShapes191Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 170 OUT1 178 |
| ✓ | BbhGeometricShapes192Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT2 546 |
| ✓ | BbhGeometricShapes193BmThis SVG path element <path d="M 41.63,86.23 L 86.15,83.17"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT245 |
| × | BbhGeometricShapes194Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(K)")
Context: BbhGeometricShapes194Bm: expected (K), got [h]
Values:
got = "h" |
Failed | IN1 095 OUT3 266 |
| ✓ | BbhGeometricShapes195Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 666 |
| ✓ | BbhGeometricShapes196Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 525 |
| ✓ | BbhGeometricShapes197Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT783 |
| ✓ | BbhGeometricShapes198Bmidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 188 |
Reasoning - BBH geometric shapes (SVG paths) (set 7)
22 passed10 failed |
68.8% |
IN35 172 OUT46 465 | |
| × | BbhGeometricShapes199BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes199BN: expected (K), got [(H)] |
Failed | IN1 086 OUT2 590 |
| × | BbhGeometricShapes200BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes200BN: expected (K), got [(A)] |
Failed | IN1 101 OUT1 421 |
| ✓ | BbhGeometricShapes201BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT870 |
| ✓ | BbhGeometricShapes202BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT1 053 |
| × | BbhGeometricShapes203BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes203BN: expected (K), got [(A)] |
Failed | IN1 099 OUT902 |
| ✓ | BbhGeometricShapes204BNThis SVG path element <path d="M 68.00,36.00 L 61.00,80.00"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT246 |
| ✓ | BbhGeometricShapes205BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 141 OUT1 438 |
| ✓ | BbhGeometricShapes206BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 079 |
| ✓ | BbhGeometricShapes207BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT1 835 |
| ✓ | BbhGeometricShapes208BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT1 398 |
| ✓ | BbhGeometricShapes209BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT943 |
| ✓ | BbhGeometricShapes210BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 655 |
| × | BbhGeometricShapes211BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes211BN: expected (K), got [(H)] |
Failed | IN1 125 OUT2 664 |
| × | BbhGeometricShapes212BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(B)")
Context: BbhGeometricShapes212BN: expected (B), got [G] |
Failed | IN1 144 OUT5 843 |
| ✓ | BbhGeometricShapes213BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 502 |
| ✓ | BbhGeometricShapes214BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT673 |
| ✓ | BbhGeometricShapes215BNThis SVG path element <path d="M 37.71,30.46 L 40.10,45.62"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT249 |
| ✓ | BbhGeometricShapes216BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT2 022 |
| ✓ | BbhGeometricShapes217BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 799 |
| ✓ | BbhGeometricShapes218BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT673 |
| ✓ | BbhGeometricShapes219BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT636 |
| × | BbhGeometricShapes220BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes220BN: expected (K), got [(A)] |
Failed | IN1 099 OUT1 141 |
| × | BbhGeometricShapes221BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes221BN: expected (K), got [(A)] |
Failed | IN1 103 OUT1 101 |
| ✓ | BbhGeometricShapes222BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 748 |
| × | BbhGeometricShapes223BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes223BN: expected (K), got [(H)] |
Failed | IN1 099 OUT2 349 |
| × | BbhGeometricShapes224BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes224BN: expected (K), got [(H)] |
Failed | IN1 086 OUT1 050 |
| ✓ | BbhGeometricShapes225BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT2 022 |
| × | BbhGeometricShapes226BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes226BN: expected (K), got [(H)] |
Failed | IN1 086 OUT2 189 |
| ✓ | BbhGeometricShapes227BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT576 |
| ✓ | BbhGeometricShapes228BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT905 |
| ✓ | BbhGeometricShapes229BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT541 |
| ✓ | BbhGeometricShapes230BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 352 |
Reasoning - BBH geometric shapes (SVG paths) (set 8)
29 passed3 failed |
90.6% |
IN33 751 OUT31 745 | |
| ✓ | BbhGeometricShapes231BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT971 |
| ✓ | BbhGeometricShapes232BNThis SVG path element <path d="M 93.43,69.32 L 68.06,88.58"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT311 |
| ✓ | BbhGeometricShapes233BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT1 573 |
| ✓ | BbhGeometricShapes234BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT714 |
| ✓ | BbhGeometricShapes235BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 085 |
| ✓ | BbhGeometricShapes236BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 679 |
| ✓ | BbhGeometricShapes237BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT969 |
| × | BbhGeometricShapes238BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes238BN: expected (K), got [(H)] |
Failed | IN1 086 OUT1 007 |
| ✓ | BbhGeometricShapes239BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT796 |
| ✓ | BbhGeometricShapes240BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT732 |
| ✓ | BbhGeometricShapes241BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT1 126 |
| × | BbhGeometricShapes242BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes242BN: expected (K), got [(A)] |
Failed | IN1 103 OUT1 513 |
| ✓ | BbhGeometricShapes243BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT1 666 |
| ✓ | BbhGeometricShapes244BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT603 |
| ✓ | BbhGeometricShapes245BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 641 |
| ✓ | BbhGeometricShapes246BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT2 895 |
| × | BbhGeometricShapes247BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapes247BN: expected (K), got [(H)] |
Failed | IN1 095 OUT2 238 |
| ✓ | BbhGeometricShapes248BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT2 376 |
| ✓ | BbhGeometricShapes249BNidentify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 807 |
| ✓ | BbhHyperbaton10BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT419 |
| ✓ | BbhHyperbaton11BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT425 |
| ✓ | BbhHyperbaton12BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT700 |
| ✓ | BbhHyperbaton13BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT585 |
| ✓ | BbhHyperbaton14BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT297 |
| ✓ | BbhHyperbaton15BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT484 |
| ✓ | BbhHyperbaton16BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT364 |
| ✓ | BbhHyperbaton17BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT385 |
| ✓ | BbhHyperbaton18BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT388 |
| ✓ | BbhHyperbaton19BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT419 |
| ✓ | BbhHyperbaton20BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT772 |
| ✓ | BbhHyperbaton8BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT459 |
| ✓ | BbhHyperbaton9BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT346 |
Reasoning - BBH hyperbaton - adjective ordering (set 1)
32 passed |
100.0% |
IN31 506 OUT17 369 | |
| ✓ | BbhHyperbaton21BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT388 |
| ✓ | BbhHyperbaton22BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 160 |
| ✓ | BbhHyperbaton23BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT369 |
| ✓ | BbhHyperbaton24BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT381 |
| ✓ | BbhHyperbaton25BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT407 |
| ✓ | BbhHyperbaton26BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT425 |
| ✓ | BbhHyperbaton27BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT432 |
| ✓ | BbhHyperbaton28BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT435 |
| ✓ | BbhHyperbaton29BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT855 |
| ✓ | BbhHyperbaton30BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT432 |
| ✓ | BbhHyperbaton31BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT397 |
| ✓ | BbhHyperbaton32BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT275 |
| ✓ | BbhHyperbaton33BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT403 |
| ✓ | BbhHyperbaton34BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT507 |
| ✓ | BbhHyperbaton35BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 174 |
| ✓ | BbhHyperbaton36BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT392 |
| ✓ | BbhHyperbaton37BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT372 |
| ✓ | BbhHyperbaton38BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 067 |
| ✓ | BbhHyperbaton39BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT310 |
| ✓ | BbhHyperbaton40BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT426 |
| ✓ | BbhHyperbaton41BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT432 |
| ✓ | BbhHyperbaton42BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT348 |
| ✓ | BbhHyperbaton43BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT889 |
| ✓ | BbhHyperbaton44BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT661 |
| ✓ | BbhHyperbaton45BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT441 |
| ✓ | BbhHyperbaton46BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT349 |
| ✓ | BbhHyperbaton47BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 058 |
| ✓ | BbhHyperbaton48BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT826 |
| ✓ | BbhHyperbaton49BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT418 |
| ✓ | BbhHyperbaton50BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT717 |
| ✓ | BbhHyperbaton51BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT258 |
| ✓ | BbhHyperbaton52BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT365 |
Reasoning - BBH hyperbaton - adjective ordering (set 2)
32 passed |
100.0% |
IN31 524 OUT18 903 | |
| ✓ | BbhHyperbaton53BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT442 |
| ✓ | BbhHyperbaton54BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT533 |
| ✓ | BbhHyperbaton55BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT359 |
| ✓ | BbhHyperbaton56BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT857 |
| ✓ | BbhHyperbaton57BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT1 756 |
| ✓ | BbhHyperbaton58BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT816 |
| ✓ | BbhHyperbaton59BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT487 |
| ✓ | BbhHyperbaton60BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT659 |
| ✓ | BbhHyperbaton61BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT394 |
| ✓ | BbhHyperbaton62BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT347 |
| ✓ | BbhHyperbaton63BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT309 |
| ✓ | BbhHyperbaton64BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT357 |
| ✓ | BbhHyperbaton65BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT386 |
| ✓ | BbhHyperbaton66BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT457 |
| ✓ | BbhHyperbaton67BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT422 |
| ✓ | BbhHyperbaton68BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT376 |
| ✓ | BbhHyperbaton69BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT506 |
| ✓ | BbhHyperbaton70BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT354 |
| ✓ | BbhHyperbaton71BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT352 |
| ✓ | BbhHyperbaton72BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT752 |
| ✓ | BbhHyperbaton73BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT460 |
| ✓ | BbhHyperbaton74BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT440 |
| ✓ | BbhHyperbaton75BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT490 |
| ✓ | BbhHyperbaton76BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT417 |
| ✓ | BbhHyperbaton77BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT400 |
| ✓ | BbhHyperbaton78BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT442 |
| ✓ | BbhHyperbaton79BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 373 |
| ✓ | BbhHyperbaton80BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT395 |
| ✓ | BbhHyperbaton81BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 093 |
| ✓ | BbhHyperbaton82BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT375 |
| ✓ | BbhHyperbaton83BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT712 |
| ✓ | BbhHyperbaton84BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 385 |
Reasoning - BBH hyperbaton - adjective ordering (set 3)
30 passed |
100.0% |
IN29 554 OUT14 691 | |
| ✓ | BbhHyperbaton100BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT961 |
| ✓ | BbhHyperbaton101BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT427 |
| ✓ | BbhHyperbaton102BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT596 |
| ✓ | BbhHyperbaton103BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT459 |
| ✓ | BbhHyperbaton104BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT442 |
| ✓ | BbhHyperbaton105BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT448 |
| ✓ | BbhHyperbaton106BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT425 |
| ✓ | BbhHyperbaton107BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT417 |
| ✓ | BbhHyperbaton108BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT358 |
| ✓ | BbhHyperbaton109BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT356 |
| ✓ | BbhHyperbaton110BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT565 |
| ✓ | BbhHyperbaton111BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT464 |
| ✓ | BbhHyperbaton112BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT296 |
| ✓ | BbhHyperbaton113BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT364 |
| ✓ | BbhHyperbaton114BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT395 |
| ✓ | BbhHyperbaton85BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT730 |
| ✓ | BbhHyperbaton86BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT847 |
| ✓ | BbhHyperbaton87BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT555 |
| ✓ | BbhHyperbaton88BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT461 |
| ✓ | BbhHyperbaton89BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT336 |
| ✓ | BbhHyperbaton90BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT427 |
| ✓ | BbhHyperbaton91BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT616 |
| ✓ | BbhHyperbaton92BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT419 |
| ✓ | BbhHyperbaton93BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT419 |
| ✓ | BbhHyperbaton94BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT356 |
| ✓ | BbhHyperbaton95BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT420 |
| ✓ | BbhHyperbaton96BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT780 |
| ✓ | BbhHyperbaton97BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT698 |
| ✓ | BbhHyperbaton98BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT297 |
| ✓ | BbhHyperbaton99BNWhich sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT357 |
Reasoning - BBH hyperbaton - adjective ordering (set 4)
32 passed |
100.0% |
IN31 490 OUT17 815 | |
| ✓ | BbhHyperbaton115BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT390 |
| ✓ | BbhHyperbaton116BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT510 |
| ✓ | BbhHyperbaton117BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT410 |
| ✓ | BbhHyperbaton118BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT288 |
| ✓ | BbhHyperbaton119BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT892 |
| ✓ | BbhHyperbaton120BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT264 |
| ✓ | BbhHyperbaton121BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT393 |
| ✓ | BbhHyperbaton122BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT522 |
| ✓ | BbhHyperbaton123BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT456 |
| ✓ | BbhHyperbaton124BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT532 |
| ✓ | BbhHyperbaton125BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT259 |
| ✓ | BbhHyperbaton126BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 385 |
| ✓ | BbhHyperbaton127BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT306 |
| ✓ | BbhHyperbaton128BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT309 |
| ✓ | BbhHyperbaton129BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT356 |
| ✓ | BbhHyperbaton130BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT467 |
| ✓ | BbhHyperbaton131BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 470 |
| ✓ | BbhHyperbaton132BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 341 |
| ✓ | BbhHyperbaton133BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT434 |
| ✓ | BbhHyperbaton134BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT518 |
| ✓ | BbhHyperbaton135BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT288 |
| ✓ | BbhHyperbaton136BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT448 |
| ✓ | BbhHyperbaton137BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT464 |
| ✓ | BbhHyperbaton138BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT493 |
| ✓ | BbhHyperbaton139BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 334 |
| ✓ | BbhHyperbaton140BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT360 |
| ✓ | BbhHyperbaton141BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT356 |
| ✓ | BbhHyperbaton142BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT400 |
| ✓ | BbhHyperbaton143BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT410 |
| ✓ | BbhHyperbaton144BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT380 |
| ✓ | BbhHyperbaton145BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT428 |
| ✓ | BbhHyperbaton146BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT952 |
Reasoning - BBH hyperbaton - adjective ordering (set 5)
32 passed |
100.0% |
IN31 512 OUT15 318 | |
| ✓ | BbhHyperbaton147BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT660 |
| ✓ | BbhHyperbaton148BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT456 |
| ✓ | BbhHyperbaton149BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT1 059 |
| ✓ | BbhHyperbaton150BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT459 |
| ✓ | BbhHyperbaton151BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT376 |
| ✓ | BbhHyperbaton152BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT364 |
| ✓ | BbhHyperbaton153BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT402 |
| ✓ | BbhHyperbaton154BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT683 |
| ✓ | BbhHyperbaton155BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT740 |
| ✓ | BbhHyperbaton156BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT355 |
| ✓ | BbhHyperbaton157BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT292 |
| ✓ | BbhHyperbaton158BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT473 |
| ✓ | BbhHyperbaton159BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT399 |
| ✓ | BbhHyperbaton160BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT287 |
| ✓ | BbhHyperbaton161BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT485 |
| ✓ | BbhHyperbaton162BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT482 |
| ✓ | BbhHyperbaton163BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT320 |
| ✓ | BbhHyperbaton164BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT393 |
| ✓ | BbhHyperbaton165BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT287 |
| ✓ | BbhHyperbaton166BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT441 |
| ✓ | BbhHyperbaton167BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT474 |
| ✓ | BbhHyperbaton168BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT753 |
| ✓ | BbhHyperbaton169BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT335 |
| ✓ | BbhHyperbaton170BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT446 |
| ✓ | BbhHyperbaton171BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT483 |
| ✓ | BbhHyperbaton172BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT717 |
| ✓ | BbhHyperbaton173BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT401 |
| ✓ | BbhHyperbaton174BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT350 |
| ✓ | BbhHyperbaton175BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT713 |
| ✓ | BbhHyperbaton176BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT339 |
| ✓ | BbhHyperbaton177BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT487 |
| ✓ | BbhHyperbaton178BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT407 |
Reasoning - BBH hyperbaton - adjective ordering (set 6)
32 passed |
100.0% |
IN31 500 OUT16 381 | |
| ✓ | BbhHyperbaton179BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT872 |
| ✓ | BbhHyperbaton180BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT353 |
| ✓ | BbhHyperbaton181BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT802 |
| ✓ | BbhHyperbaton182BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT602 |
| ✓ | BbhHyperbaton183BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT751 |
| ✓ | BbhHyperbaton184BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT477 |
| ✓ | BbhHyperbaton185BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT387 |
| ✓ | BbhHyperbaton186BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 606 |
| ✓ | BbhHyperbaton187BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT755 |
| ✓ | BbhHyperbaton188BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT384 |
| ✓ | BbhHyperbaton189BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT344 |
| ✓ | BbhHyperbaton190BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT364 |
| ✓ | BbhHyperbaton191BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT307 |
| ✓ | BbhHyperbaton192BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT337 |
| ✓ | BbhHyperbaton193BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT375 |
| ✓ | BbhHyperbaton194BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT362 |
| ✓ | BbhHyperbaton195BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT499 |
| ✓ | BbhHyperbaton196BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT383 |
| ✓ | BbhHyperbaton197BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT414 |
| ✓ | BbhHyperbaton198BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT493 |
| ✓ | BbhHyperbaton199BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT446 |
| ✓ | BbhHyperbaton200BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT397 |
| ✓ | BbhHyperbaton201BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT314 |
| ✓ | BbhHyperbaton202BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT431 |
| ✓ | BbhHyperbaton203BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT662 |
| ✓ | BbhHyperbaton204BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT399 |
| ✓ | BbhHyperbaton205BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT408 |
| ✓ | BbhHyperbaton206BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT834 |
| ✓ | BbhHyperbaton207BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT587 |
| ✓ | BbhHyperbaton208BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT387 |
| ✓ | BbhHyperbaton209BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT299 |
| ✓ | BbhHyperbaton210BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT350 |
Reasoning - BBH hyperbaton - adjective ordering (set 7)
32 passed |
100.0% |
IN31 486 OUT16 312 | |
| ✓ | BbhHyperbaton211BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT758 |
| ✓ | BbhHyperbaton212BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT393 |
| ✓ | BbhHyperbaton213BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT545 |
| ✓ | BbhHyperbaton214BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT403 |
| ✓ | BbhHyperbaton215BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT1 078 |
| ✓ | BbhHyperbaton216BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT384 |
| ✓ | BbhHyperbaton217BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT560 |
| ✓ | BbhHyperbaton218BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT543 |
| ✓ | BbhHyperbaton219BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT386 |
| ✓ | BbhHyperbaton220BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT270 |
| ✓ | BbhHyperbaton221BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT265 |
| ✓ | BbhHyperbaton222BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT834 |
| ✓ | BbhHyperbaton223BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT402 |
| ✓ | BbhHyperbaton224BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT509 |
| ✓ | BbhHyperbaton225BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT494 |
| ✓ | BbhHyperbaton226BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT893 |
| ✓ | BbhHyperbaton227BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT398 |
| ✓ | BbhHyperbaton228BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT447 |
| ✓ | BbhHyperbaton229BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT761 |
| ✓ | BbhHyperbaton230BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT426 |
| ✓ | BbhHyperbaton231BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT441 |
| ✓ | BbhHyperbaton232BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT380 |
| ✓ | BbhHyperbaton233BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT326 |
| ✓ | BbhHyperbaton234BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT352 |
| ✓ | BbhHyperbaton235BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT371 |
| ✓ | BbhHyperbaton236BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT306 |
| ✓ | BbhHyperbaton237BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT389 |
| ✓ | BbhHyperbaton238BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT325 |
| ✓ | BbhHyperbaton239BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT745 |
| ✓ | BbhHyperbaton240BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT837 |
| ✓ | BbhHyperbaton241BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT427 |
| ✓ | BbhHyperbaton242BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT664 |
Reasoning - BBH logic: boolean expressions, causal judgement, dates
20 passed4 failed |
83.3% |
IN25 895 OUT12 722 | |
| ✓ | BbhBooleanExpressionsBa0not ( True ) and ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT220 |
| ✓ | BbhBooleanExpressionsBa1True and not not ( not False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT253 |
| ✓ | BbhBooleanExpressionsBa2not True or False or ( False ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT319 |
| ✓ | BbhBooleanExpressionsBa3False or not ( True ) and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT310 |
| × | BbhBooleanExpressionsBa4True or not False and True and False is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("True")
Context: BbhBooleanExpressionsBa4: expected True, got [False] |
Failed | IN964 OUT290 |
| ✓ | BbhBooleanExpressionsBa5False or not not not False and True is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT357 |
| ✓ | BbhBooleanExpressionsBa6not True and ( False or True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT287 |
| ✓ | BbhBooleanExpressionsBa7True and not False or ( True ) is1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN964 OUT320 |
| ✓ | BbhCausalJudgementBa0judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 019 |
| ✓ | BbhCausalJudgementBa1judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 350 OUT1 700 |
| ✓ | BbhCausalJudgementBa2judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 352 OUT634 |
| × | BbhCausalJudgementBa3judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("No")
Context: BbhCausalJudgementBa3: expected No, got [Yes] |
Failed | IN1 345 OUT1 014 |
| ✓ | BbhCausalJudgementBa4judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT652 |
| ✓ | BbhCausalJudgementBa5judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT1 075 |
| ✓ | BbhCausalJudgementBa6judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT295 |
| ✓ | BbhCausalJudgementBa7judge causation in a short scenario1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT570 |
| ✓ | BbhDateUnderstandingBa0date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT222 |
| ✓ | BbhDateUnderstandingBa1date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT456 |
| × | BbhDateUnderstandingBa2date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(B)")
Context: BbhDateUnderstandingBa2: expected (B), got [(C)] |
Failed | IN1 079 OUT488 |
| ✓ | BbhDateUnderstandingBa3date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT386 |
| ✓ | BbhDateUnderstandingBa4date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT350 |
| ✓ | BbhDateUnderstandingBa5date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT485 |
| × | BbhDateUnderstandingBa6date arithmetic / understanding1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhDateUnderstandingBa6: expected (A), got [(D)] |
Failed | IN1 087 OUT788 |
| ✓ | BbhDateUnderstandingBa7Tomorrow is 11/12/2019. What is the date yesterday in MM/DD/YYYY?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT232 |
Reasoning - BBH logic: disambiguation, dyck languages, formal fallacies
20 passed4 failed |
83.3% |
IN24 625 OUT23 880 | |
| ✓ | BbhDisambiguationQaBa0resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT352 |
| ✓ | BbhDisambiguationQaBa1resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT481 |
| × | BbhDisambiguationQaBa2resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhDisambiguationQaBa2: expected (C), got [(B)] |
Failed | IN1 026 OUT686 |
| ✓ | BbhDisambiguationQaBa3resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT384 |
| ✓ | BbhDisambiguationQaBa4resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT874 |
| × | BbhDisambiguationQaBa5resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhDisambiguationQaBa5: expected (C), got [(B)] |
Failed | IN1 023 OUT2 185 |
| × | BbhDisambiguationQaBa6resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhDisambiguationQaBa6: expected (C), got [(A)] |
Failed | IN1 027 OUT509 |
| ✓ | BbhDisambiguationQaBa7resolve pronoun antecedent ambiguity1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT510 |
| × | BbhDyckLanguagesBa0correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: NormBrackets(got) == NormBrackets("] ]")
Context: BbhDyckLanguagesBa0: expected ] ], got [[ [ ] ]] |
Failed | IN975 OUT146 |
| ✓ | BbhDyckLanguagesBa1correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT721 |
| ✓ | BbhDyckLanguagesBa2correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 038 OUT7 223 |
| ✓ | BbhDyckLanguagesBa3correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT629 |
| ✓ | BbhDyckLanguagesBa4correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT857 |
| ✓ | BbhDyckLanguagesBa5correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT842 |
| ✓ | BbhDyckLanguagesBa6correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT225 |
| ✓ | BbhDyckLanguagesBa7correctly close a sequence of brackets1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT620 |
| ✓ | BbhFormalFallaciesBa0decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT615 |
| ✓ | BbhFormalFallaciesBa1decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT688 |
| ✓ | BbhFormalFallaciesBa2decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT1 195 |
| ✓ | BbhFormalFallaciesBa3decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT789 |
| ✓ | BbhFormalFallaciesBa4decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT1 106 |
| ✓ | BbhFormalFallaciesBa5decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT974 |
| ✓ | BbhFormalFallaciesBa6decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT644 |
| ✓ | BbhFormalFallaciesBa7decide if an argument is valid or a fallacy1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT625 |
Reasoning - BBH logic: geometric shapes, hyperbaton, logical deduction
22 passed2 failed |
91.7% |
IN25 466 OUT24 078 | |
| ✓ | BbhGeometricShapesBa0identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT2 131 |
| ✓ | BbhGeometricShapesBa1identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT642 |
| × | BbhGeometricShapesBa2identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapesBa2: expected (K), got [(H)] |
Failed | IN1 085 OUT2 541 |
| ✓ | BbhGeometricShapesBa3This SVG path element <path d="M 12.64,60.32 L 16.36,27.35"/> draws a1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT302 |
| × | BbhGeometricShapesBa4identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(K)")
Context: BbhGeometricShapesBa4: expected (K), got [(H)] |
Failed | IN1 112 OUT2 292 |
| ✓ | BbhGeometricShapesBa5identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT1 608 |
| ✓ | BbhGeometricShapesBa6identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT605 |
| ✓ | BbhGeometricShapesBa7identify the shape drawn by an SVG path1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT1 409 |
| ✓ | BbhHyperbatonBa0Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT411 |
| ✓ | BbhHyperbatonBa1Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT405 |
| ✓ | BbhHyperbatonBa2Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT840 |
| ✓ | BbhHyperbatonBa3Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT973 |
| ✓ | BbhHyperbatonBa4Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 337 |
| ✓ | BbhHyperbatonBa5Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT359 |
| ✓ | BbhHyperbatonBa6Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT473 |
| ✓ | BbhHyperbatonBa7Which sentence has the correct adjective order:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT365 |
| ✓ | BbhLogicalDeductionFiveBa0order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT472 |
| ✓ | BbhLogicalDeductionFiveBa1order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 025 |
| ✓ | BbhLogicalDeductionFiveBa2order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT813 |
| ✓ | BbhLogicalDeductionFiveBa3order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT484 |
| ✓ | BbhLogicalDeductionFiveBa4order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT796 |
| ✓ | BbhLogicalDeductionFiveBa5order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT1 901 |
| ✓ | BbhLogicalDeductionFiveBa6order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT1 009 |
| ✓ | BbhLogicalDeductionFiveBa7order 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT885 |
Reasoning - BBH logic: logical deduction, movie recommendation
23 passed1 failed |
95.8% |
IN25 511 OUT25 936 | |
| ✓ | BbhLogicalDeductionSeven0order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT1 356 |
| × | BbhLogicalDeductionSeven1order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhLogicalDeductionSeven1: expected (B), got [c]
Values:
got = "c" |
Failed | IN1 165 OUT8 175 |
| ✓ | BbhLogicalDeductionSeven2order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT994 |
| ✓ | BbhLogicalDeductionSeven3order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT913 |
| ✓ | BbhLogicalDeductionSeven4order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT938 |
| ✓ | BbhLogicalDeductionSeven5order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 097 OUT1 448 |
| ✓ | BbhLogicalDeductionSeven6order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT602 |
| ✓ | BbhLogicalDeductionSeven7order 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 142 OUT1 142 |
| ✓ | BbhLogicalDeductionThree0order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT372 |
| ✓ | BbhLogicalDeductionThree1order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT257 |
| ✓ | BbhLogicalDeductionThree2order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT320 |
| ✓ | BbhLogicalDeductionThree3order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT323 |
| ✓ | BbhLogicalDeductionThree4order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT456 |
| ✓ | BbhLogicalDeductionThree5order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT374 |
| ✓ | BbhLogicalDeductionThree6order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT268 |
| ✓ | BbhLogicalDeductionThree7order 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT220 |
| ✓ | BbhMovieRec0Find a movie similar to Batman, The Mask, The Fugitive, Pretty Woman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT713 |
| ✓ | BbhMovieRec1pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT996 |
| ✓ | BbhMovieRec2pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT711 |
| ✓ | BbhMovieRec3pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 059 |
| ✓ | BbhMovieRec4Find a movie similar to Braveheart, The Mask, The Fugitive, Batman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 159 |
| ✓ | BbhMovieRec5pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT768 |
| ✓ | BbhMovieRec6pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT631 |
| ✓ | BbhMovieRec7pick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT741 |
Reasoning - BBH logic: multistep arithmetic, navigation, object counting
24 passed |
100.0% |
IN23 808 OUT9 637 | |
| ✓ | BbhMultistepArith0((-1 + 2 + 9 * 5) - (-2 + -4 + -4 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT395 |
| ✓ | BbhMultistepArith1((-9 * -5 - 6 + -2) - (-8 - -6 * -3 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT610 |
| ✓ | BbhMultistepArith2((3 * -3 * 6 + -5) - (-2 + -7 - 7 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT359 |
| ✓ | BbhMultistepArith3((6 * -6 * 8 * 1) * (-1 * 7 * -6 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT485 |
| ✓ | BbhMultistepArith4((-6 - -4 + 9 + 0) + (1 + -4 - -9 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT981 |
| ✓ | BbhMultistepArith5((-6 - 4 * 2 - 6) + (1 + -2 * 1 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT395 |
| ✓ | BbhMultistepArith6((1 - 0 + 1 - 4) - (-3 * 1 - -6 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT364 |
| ✓ | BbhMultistepArith7((1 + 7 * -9 + -5) + (3 + -5 * 2 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT388 |
| ✓ | BbhNavigate0does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT1 678 |
| ✓ | BbhNavigate1does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT618 |
| ✓ | BbhNavigate2does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT398 |
| ✓ | BbhNavigate3does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT197 |
| ✓ | BbhNavigate4does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT426 |
| ✓ | BbhNavigate5does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT165 |
| ✓ | BbhNavigate6does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT668 |
| ✓ | BbhNavigate7does a navigation path return to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT171 |
| ✓ | BbhObjectCount0count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT197 |
| ✓ | BbhObjectCount1count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT298 |
| ✓ | BbhObjectCount2I have a fridge, a chair, and a microwave. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT52 |
| ✓ | BbhObjectCount3count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT354 |
| ✓ | BbhObjectCount4count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT79 |
| ✓ | BbhObjectCount5I have a fish, a bear, and a frog. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT68 |
| ✓ | BbhObjectCount6I have a car, and a toaster. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT40 |
| ✓ | BbhObjectCount7count objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT251 |
Reasoning - BBH logic: object tracking, web of lies, word sorting
23 passed1 failed |
95.8% |
IN24 430 OUT9 028 | |
| ✓ | BbhTrackThree0track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT316 |
| ✓ | BbhTrackThree1track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT376 |
| ✓ | BbhTrackThree2track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT319 |
| ✓ | BbhTrackThree3track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT431 |
| ✓ | BbhTrackThree4track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT300 |
| ✓ | BbhTrackThree5track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT324 |
| ✓ | BbhTrackThree6track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT369 |
| ✓ | BbhTrackThree7track 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT383 |
| ✓ | BbhWebLies0evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT256 |
| ✓ | BbhWebLies1evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT283 |
| ✓ | BbhWebLies2evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT266 |
| ✓ | BbhWebLies3evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT239 |
| ✓ | BbhWebLies4evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT315 |
| ✓ | BbhWebLies5evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT264 |
| ✓ | BbhWebLies6evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT467 |
| ✓ | BbhWebLies7evaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT247 |
| ✓ | BbhWordSort0Sort the following words alphabetically: List: syndrome therefrom1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT102 |
| ✓ | BbhWordSort1sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT807 |
| × | BbhWordSort2sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("artillery bainite doris fda harm incongruous monkey prosody vegetate vivian")
Context: BbhWordSort2: expected sorted list, got [bainite doris fda harm incongruous monkey prosody vegetate vivian] |
Failed | IN982 OUT436 |
| ✓ | BbhWordSort3sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT287 |
| ✓ | BbhWordSort4sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT136 |
| ✓ | BbhWordSort5sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT703 |
| ✓ | BbhWordSort6sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT834 |
| ✓ | BbhWordSort7sort a list of words alphabetically1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT568 |
Reasoning - BBH logic: penguins table, colored objects, ruin names
24 passed |
100.0% |
IN26 008 OUT10 711 | |
| ✓ | BbhColoredObjects0reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT283 |
| ✓ | BbhColoredObjects1reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT235 |
| ✓ | BbhColoredObjects2reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT141 |
| ✓ | BbhColoredObjects3reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 041 OUT439 |
| ✓ | BbhColoredObjects4reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT164 |
| ✓ | BbhColoredObjects5reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT304 |
| ✓ | BbhColoredObjects6reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT255 |
| ✓ | BbhColoredObjects7reason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT168 |
| ✓ | BbhPenguins0answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 217 OUT176 |
| ✓ | BbhPenguins1answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT142 |
| ✓ | BbhPenguins2answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT251 |
| ✓ | BbhPenguins3answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 201 OUT368 |
| ✓ | BbhPenguins4answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT214 |
| ✓ | BbhPenguins5answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 207 OUT196 |
| ✓ | BbhPenguins6answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 226 OUT275 |
| ✓ | BbhPenguins7answer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 202 OUT316 |
| ✓ | BbhRuinNames0pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT827 |
| ✓ | BbhRuinNames1pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT382 |
| ✓ | BbhRuinNames2pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT446 |
| ✓ | BbhRuinNames3pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT375 |
| ✓ | BbhRuinNames4pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 772 |
| ✓ | BbhRuinNames5pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT1 476 |
| ✓ | BbhRuinNames6pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT486 |
| ✓ | BbhRuinNames7pick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 020 |
Reasoning - BBH logic: temporal sequences, object tracking
24 passed |
100.0% |
IN27 227 OUT15 175 | |
| ✓ | BbhTemporal0find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT379 |
| ✓ | BbhTemporal1find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT462 |
| ✓ | BbhTemporal2find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT430 |
| ✓ | BbhTemporal3find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 123 OUT416 |
| ✓ | BbhTemporal4find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT480 |
| ✓ | BbhTemporal5find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT547 |
| ✓ | BbhTemporal6find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 137 OUT433 |
| ✓ | BbhTemporal7find the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT514 |
| ✓ | BbhTrackFive0track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT580 |
| ✓ | BbhTrackFive1track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT994 |
| ✓ | BbhTrackFive2track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT691 |
| ✓ | BbhTrackFive3track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT655 |
| ✓ | BbhTrackFive4track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT564 |
| ✓ | BbhTrackFive5track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT471 |
| ✓ | BbhTrackFive6track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT552 |
| ✓ | BbhTrackFive7track 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT483 |
| ✓ | BbhTrackSeven0track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT578 |
| ✓ | BbhTrackSeven1track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT756 |
| ✓ | BbhTrackSeven2track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT934 |
| ✓ | BbhTrackSeven3track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT873 |
| ✓ | BbhTrackSeven4track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT878 |
| ✓ | BbhTrackSeven5track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT901 |
| ✓ | BbhTrackSeven6track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT823 |
| ✓ | BbhTrackSeven7track 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT781 |
Reasoning - BBH logic: translation errors, snarks, sports
19 passed5 failed |
79.2% |
IN25 261 OUT15 623 | |
| ✓ | BbhSalientTransErr0identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 188 OUT656 |
| × | BbhSalientTransErr1identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhSalientTransErr1: expected (A), got [(D)] |
Failed | IN1 188 OUT348 |
| × | BbhSalientTransErr2identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(F)")
Context: BbhSalientTransErr2: expected (F), got [(D)] |
Failed | IN1 185 OUT992 |
| × | BbhSalientTransErr3identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(F)")
Context: BbhSalientTransErr3: expected (F), got [(D)] |
Failed | IN1 182 OUT1 099 |
| × | BbhSalientTransErr4identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhSalientTransErr4: expected (A), got [(F)] |
Failed | IN1 179 OUT3 339 |
| ✓ | BbhSalientTransErr5identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 191 OUT1 623 |
| ✓ | BbhSalientTransErr6identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 201 OUT542 |
| ✓ | BbhSalientTransErr7identify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 166 OUT1 125 |
| ✓ | BbhSnarks0Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT402 |
| ✓ | BbhSnarks1Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT342 |
| ✓ | BbhSnarks2Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT329 |
| ✓ | BbhSnarks3Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT383 |
| ✓ | BbhSnarks4Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT827 |
| ✓ | BbhSnarks5Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT481 |
| ✓ | BbhSnarks6Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT294 |
| ✓ | BbhSnarks7Which statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT401 |
| × | BbhSports0Is the following sentence plausible? "Elias Lindholm beat the buzzer."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("no")
Context: BbhSports0: expected no, got [Yes] |
Failed | IN971 OUT290 |
| ✓ | BbhSports1judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN970 OUT290 |
| ✓ | BbhSports2judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN974 OUT332 |
| ✓ | BbhSports3judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN978 OUT261 |
| ✓ | BbhSports4judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN972 OUT364 |
| ✓ | BbhSports5judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN978 OUT277 |
| ✓ | BbhSports6Is the following sentence plausible? "David Silva took a throw in."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN969 OUT323 |
| ✓ | BbhSports7judge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN972 OUT303 |
Reasoning - BBH logical deduction - order 3 objects (set 1)
32 passed |
100.0% |
IN33 651 OUT11 897 | |
| ✓ | BbhLogicalDeductionThree21BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT426 |
| ✓ | BbhLogicalDeductionThree22BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT316 |
| ✓ | BbhLogicalDeductionThree23BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT257 |
| ✓ | BbhLogicalDeductionThree24BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT217 |
| ✓ | BbhLogicalDeductionThree25BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT235 |
| ✓ | BbhLogicalDeductionThree26BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT325 |
| ✓ | BbhLogicalDeductionThree27BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT453 |
| ✓ | BbhLogicalDeductionThree28BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT549 |
| ✓ | BbhLogicalDeductionThree29BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT409 |
| ✓ | BbhLogicalDeductionThree30BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT243 |
| ✓ | BbhLogicalDeductionThree31BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT279 |
| ✓ | BbhLogicalDeductionThree32BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT598 |
| ✓ | BbhLogicalDeductionThree33BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT405 |
| ✓ | BbhLogicalDeductionThree34BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT747 |
| ✓ | BbhLogicalDeductionThree35BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT242 |
| ✓ | BbhLogicalDeductionThree36BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT308 |
| ✓ | BbhLogicalDeductionThree37BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT414 |
| ✓ | BbhLogicalDeductionThree38BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT456 |
| ✓ | BbhLogicalDeductionThree39BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT332 |
| ✓ | BbhLogicalDeductionThree40BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT333 |
| ✓ | BbhLogicalDeductionThree41BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT308 |
| ✓ | BbhLogicalDeductionThree42BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT262 |
| ✓ | BbhLogicalDeductionThree43BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT384 |
| ✓ | BbhLogicalDeductionThree44BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT350 |
| ✓ | BbhLogicalDeductionThree45BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT336 |
| ✓ | BbhLogicalDeductionThree46BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT799 |
| ✓ | BbhLogicalDeductionThree47BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT272 |
| ✓ | BbhLogicalDeductionThree48BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT401 |
| ✓ | BbhLogicalDeductionThree49BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT406 |
| ✓ | BbhLogicalDeductionThree50BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT249 |
| ✓ | BbhLogicalDeductionThree51BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT266 |
| ✓ | BbhLogicalDeductionThree52BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT320 |
Reasoning - BBH logical deduction - order 3 objects (set 2)
32 passed |
100.0% |
IN33 500 OUT11 393 | |
| ✓ | BbhLogicalDeductionThree53BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT661 |
| ✓ | BbhLogicalDeductionThree54BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT334 |
| ✓ | BbhLogicalDeductionThree55BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT274 |
| ✓ | BbhLogicalDeductionThree56BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT248 |
| ✓ | BbhLogicalDeductionThree57BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT342 |
| ✓ | BbhLogicalDeductionThree58BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT322 |
| ✓ | BbhLogicalDeductionThree59BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT419 |
| ✓ | BbhLogicalDeductionThree60BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT724 |
| ✓ | BbhLogicalDeductionThree61BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT343 |
| ✓ | BbhLogicalDeductionThree62BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT463 |
| ✓ | BbhLogicalDeductionThree63BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT386 |
| ✓ | BbhLogicalDeductionThree64BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT306 |
| ✓ | BbhLogicalDeductionThree65BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT580 |
| ✓ | BbhLogicalDeductionThree66BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT391 |
| ✓ | BbhLogicalDeductionThree67BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT209 |
| ✓ | BbhLogicalDeductionThree68BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT332 |
| ✓ | BbhLogicalDeductionThree69BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT353 |
| ✓ | BbhLogicalDeductionThree70BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT315 |
| ✓ | BbhLogicalDeductionThree71BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT317 |
| ✓ | BbhLogicalDeductionThree72BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT325 |
| ✓ | BbhLogicalDeductionThree73BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT331 |
| ✓ | BbhLogicalDeductionThree74BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT267 |
| ✓ | BbhLogicalDeductionThree75BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT312 |
| ✓ | BbhLogicalDeductionThree76BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT234 |
| ✓ | BbhLogicalDeductionThree77BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT322 |
| ✓ | BbhLogicalDeductionThree78BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT298 |
| ✓ | BbhLogicalDeductionThree79BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT233 |
| ✓ | BbhLogicalDeductionThree80BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT392 |
| ✓ | BbhLogicalDeductionThree81BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT392 |
| ✓ | BbhLogicalDeductionThree82BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT298 |
| ✓ | BbhLogicalDeductionThree83BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT293 |
| ✓ | BbhLogicalDeductionThree84BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT377 |
Reasoning - BBH logical deduction - order 3 objects (set 3)
32 passed |
100.0% |
IN33 574 OUT10 999 | |
| ✓ | BbhLogicalDeductionThree100BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT322 |
| ✓ | BbhLogicalDeductionThree101BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT349 |
| ✓ | BbhLogicalDeductionThree102BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT209 |
| ✓ | BbhLogicalDeductionThree103BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT218 |
| ✓ | BbhLogicalDeductionThree104BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT297 |
| ✓ | BbhLogicalDeductionThree105BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT206 |
| ✓ | BbhLogicalDeductionThree106BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT426 |
| ✓ | BbhLogicalDeductionThree107BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT345 |
| ✓ | BbhLogicalDeductionThree108BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT313 |
| ✓ | BbhLogicalDeductionThree109BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT388 |
| ✓ | BbhLogicalDeductionThree110BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT500 |
| ✓ | BbhLogicalDeductionThree111BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT276 |
| ✓ | BbhLogicalDeductionThree112BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT332 |
| ✓ | BbhLogicalDeductionThree113BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT288 |
| ✓ | BbhLogicalDeductionThree114BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT227 |
| ✓ | BbhLogicalDeductionThree115BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT355 |
| ✓ | BbhLogicalDeductionThree116BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT593 |
| ✓ | BbhLogicalDeductionThree85BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT344 |
| ✓ | BbhLogicalDeductionThree86BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT417 |
| ✓ | BbhLogicalDeductionThree87BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT304 |
| ✓ | BbhLogicalDeductionThree88BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT378 |
| ✓ | BbhLogicalDeductionThree89BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT293 |
| ✓ | BbhLogicalDeductionThree90BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT190 |
| ✓ | BbhLogicalDeductionThree91BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT273 |
| ✓ | BbhLogicalDeductionThree92BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT304 |
| ✓ | BbhLogicalDeductionThree93BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT470 |
| ✓ | BbhLogicalDeductionThree94BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT358 |
| ✓ | BbhLogicalDeductionThree95BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT239 |
| ✓ | BbhLogicalDeductionThree96BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT491 |
| ✓ | BbhLogicalDeductionThree97BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT354 |
| ✓ | BbhLogicalDeductionThree98BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT665 |
| ✓ | BbhLogicalDeductionThree99BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT275 |
Reasoning - BBH logical deduction - order 3 objects (set 4)
32 passed |
100.0% |
IN33 622 OUT10 748 | |
| ✓ | BbhLogicalDeductionThree117BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT393 |
| ✓ | BbhLogicalDeductionThree118BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT350 |
| ✓ | BbhLogicalDeductionThree119BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT315 |
| ✓ | BbhLogicalDeductionThree120BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT202 |
| ✓ | BbhLogicalDeductionThree121BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT339 |
| ✓ | BbhLogicalDeductionThree122BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT229 |
| ✓ | BbhLogicalDeductionThree123BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT360 |
| ✓ | BbhLogicalDeductionThree124BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT314 |
| ✓ | BbhLogicalDeductionThree125BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT387 |
| ✓ | BbhLogicalDeductionThree126BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT366 |
| ✓ | BbhLogicalDeductionThree127BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT422 |
| ✓ | BbhLogicalDeductionThree128BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT313 |
| ✓ | BbhLogicalDeductionThree129BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT264 |
| ✓ | BbhLogicalDeductionThree130BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT275 |
| ✓ | BbhLogicalDeductionThree131BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT348 |
| ✓ | BbhLogicalDeductionThree132BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT239 |
| ✓ | BbhLogicalDeductionThree133BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT433 |
| ✓ | BbhLogicalDeductionThree134BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT371 |
| ✓ | BbhLogicalDeductionThree135BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT307 |
| ✓ | BbhLogicalDeductionThree136BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT468 |
| ✓ | BbhLogicalDeductionThree137BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT462 |
| ✓ | BbhLogicalDeductionThree138BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT323 |
| ✓ | BbhLogicalDeductionThree139BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT431 |
| ✓ | BbhLogicalDeductionThree140BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT300 |
| ✓ | BbhLogicalDeductionThree141BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT333 |
| ✓ | BbhLogicalDeductionThree142BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT372 |
| ✓ | BbhLogicalDeductionThree143BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT276 |
| ✓ | BbhLogicalDeductionThree144BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT356 |
| ✓ | BbhLogicalDeductionThree145BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT241 |
| ✓ | BbhLogicalDeductionThree146BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT249 |
| ✓ | BbhLogicalDeductionThree147BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT275 |
| ✓ | BbhLogicalDeductionThree148BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT435 |
Reasoning - BBH logical deduction - order 3 objects (set 5)
30 passed |
100.0% |
IN31 460 OUT11 351 | |
| ✓ | BbhLogicalDeductionThree149BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT264 |
| ✓ | BbhLogicalDeductionThree150BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT359 |
| ✓ | BbhLogicalDeductionThree151BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT309 |
| ✓ | BbhLogicalDeductionThree152BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT390 |
| ✓ | BbhLogicalDeductionThree153BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT275 |
| ✓ | BbhLogicalDeductionThree154BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT536 |
| ✓ | BbhLogicalDeductionThree155BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT610 |
| ✓ | BbhLogicalDeductionThree156BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT357 |
| ✓ | BbhLogicalDeductionThree157BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT290 |
| ✓ | BbhLogicalDeductionThree158BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT263 |
| ✓ | BbhLogicalDeductionThree159BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT440 |
| ✓ | BbhLogicalDeductionThree160BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT467 |
| ✓ | BbhLogicalDeductionThree161BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT285 |
| ✓ | BbhLogicalDeductionThree162BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT227 |
| ✓ | BbhLogicalDeductionThree163BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT350 |
| ✓ | BbhLogicalDeductionThree164BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT345 |
| ✓ | BbhLogicalDeductionThree165BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT714 |
| ✓ | BbhLogicalDeductionThree166BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT249 |
| ✓ | BbhLogicalDeductionThree167BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT359 |
| ✓ | BbhLogicalDeductionThree168BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT298 |
| ✓ | BbhLogicalDeductionThree169BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT345 |
| ✓ | BbhLogicalDeductionThree170BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT340 |
| ✓ | BbhLogicalDeductionThree171BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT475 |
| ✓ | BbhLogicalDeductionThree172BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT604 |
| ✓ | BbhLogicalDeductionThree173BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT459 |
| ✓ | BbhLogicalDeductionThree174BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT229 |
| ✓ | BbhLogicalDeductionThree175BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT367 |
| ✓ | BbhLogicalDeductionThree176BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT448 |
| ✓ | BbhLogicalDeductionThree177BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT270 |
| ✓ | BbhLogicalDeductionThree178BSorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT427 |
Reasoning - BBH logical deduction - order 3 objects (set 6)
32 passed |
100.0% |
IN33 519 OUT11 637 | |
| ✓ | BbhLogicalDeductionThreeObjects179BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT917 |
| ✓ | BbhLogicalDeductionThreeObjects180BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 044 OUT214 |
| ✓ | BbhLogicalDeductionThreeObjects181BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT463 |
| ✓ | BbhLogicalDeductionThreeObjects182BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT186 |
| ✓ | BbhLogicalDeductionThreeObjects183BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT365 |
| ✓ | BbhLogicalDeductionThreeObjects184BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT289 |
| ✓ | BbhLogicalDeductionThreeObjects185BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT258 |
| ✓ | BbhLogicalDeductionThreeObjects186BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT364 |
| ✓ | BbhLogicalDeductionThreeObjects187BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT348 |
| ✓ | BbhLogicalDeductionThreeObjects188BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT394 |
| ✓ | BbhLogicalDeductionThreeObjects189BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT307 |
| ✓ | BbhLogicalDeductionThreeObjects190BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT353 |
| ✓ | BbhLogicalDeductionThreeObjects191BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT226 |
| ✓ | BbhLogicalDeductionThreeObjects192BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT372 |
| ✓ | BbhLogicalDeductionThreeObjects193BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT268 |
| ✓ | BbhLogicalDeductionThreeObjects194BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT358 |
| ✓ | BbhLogicalDeductionThreeObjects195BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT361 |
| ✓ | BbhLogicalDeductionThreeObjects196BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT1 015 |
| ✓ | BbhLogicalDeductionThreeObjects197BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT334 |
| ✓ | BbhLogicalDeductionThreeObjects198BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT203 |
| ✓ | BbhLogicalDeductionThreeObjects199BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT378 |
| ✓ | BbhLogicalDeductionThreeObjects200BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT333 |
| ✓ | BbhLogicalDeductionThreeObjects201BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT288 |
| ✓ | BbhLogicalDeductionThreeObjects202BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT473 |
| ✓ | BbhLogicalDeductionThreeObjects203BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT386 |
| ✓ | BbhLogicalDeductionThreeObjects204BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT268 |
| ✓ | BbhLogicalDeductionThreeObjects205BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT353 |
| ✓ | BbhLogicalDeductionThreeObjects206BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT352 |
| ✓ | BbhLogicalDeductionThreeObjects207BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT343 |
| ✓ | BbhLogicalDeductionThreeObjects208BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT272 |
| ✓ | BbhLogicalDeductionThreeObjects209BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT308 |
| ✓ | BbhLogicalDeductionThreeObjects210BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT288 |
Reasoning - BBH logical deduction - order 3 objects (set 7)
32 passed |
100.0% |
IN33 552 OUT11 985 | |
| ✓ | BbhLogicalDeductionThreeObjects211BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT303 |
| ✓ | BbhLogicalDeductionThreeObjects212BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT321 |
| ✓ | BbhLogicalDeductionThreeObjects213BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT752 |
| ✓ | BbhLogicalDeductionThreeObjects214BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT444 |
| ✓ | BbhLogicalDeductionThreeObjects215BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT329 |
| ✓ | BbhLogicalDeductionThreeObjects216BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT406 |
| ✓ | BbhLogicalDeductionThreeObjects217BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT452 |
| ✓ | BbhLogicalDeductionThreeObjects218BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT319 |
| ✓ | BbhLogicalDeductionThreeObjects219BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT370 |
| ✓ | BbhLogicalDeductionThreeObjects220BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT482 |
| ✓ | BbhLogicalDeductionThreeObjects221BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT267 |
| ✓ | BbhLogicalDeductionThreeObjects222BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT389 |
| ✓ | BbhLogicalDeductionThreeObjects223BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT324 |
| ✓ | BbhLogicalDeductionThreeObjects224BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT206 |
| ✓ | BbhLogicalDeductionThreeObjects225BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT448 |
| ✓ | BbhLogicalDeductionThreeObjects226BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT416 |
| ✓ | BbhLogicalDeductionThreeObjects227BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT289 |
| ✓ | BbhLogicalDeductionThreeObjects228BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT324 |
| ✓ | BbhLogicalDeductionThreeObjects229BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT392 |
| ✓ | BbhLogicalDeductionThreeObjects230BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT202 |
| ✓ | BbhLogicalDeductionThreeObjects231BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT349 |
| ✓ | BbhLogicalDeductionThreeObjects232BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT384 |
| ✓ | BbhLogicalDeductionThreeObjects233BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT461 |
| ✓ | BbhLogicalDeductionThreeObjects234BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT469 |
| ✓ | BbhLogicalDeductionThreeObjects235BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT419 |
| ✓ | BbhLogicalDeductionThreeObjects236BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT399 |
| ✓ | BbhLogicalDeductionThreeObjects237BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT207 |
| ✓ | BbhLogicalDeductionThreeObjects238BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 049 OUT530 |
| ✓ | BbhLogicalDeductionThreeObjects239BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT282 |
| ✓ | BbhLogicalDeductionThreeObjects240BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT486 |
| ✓ | BbhLogicalDeductionThreeObjects241BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT311 |
| ✓ | BbhLogicalDeductionThreeObjects242BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT253 |
Reasoning - BBH logical deduction - order 5 objects (set 1)
30 passed |
100.0% |
IN31 910 OUT26 908 | |
| ✓ | BbhHyperbaton243BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT689 |
| ✓ | BbhHyperbaton244BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT722 |
| ✓ | BbhHyperbaton245BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT339 |
| ✓ | BbhHyperbaton246BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT335 |
| ✓ | BbhHyperbaton247BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT763 |
| ✓ | BbhHyperbaton248BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT361 |
| ✓ | BbhHyperbaton249BOchoose the correctly ordered sequence of adjectives1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT501 |
| ✓ | BbhLogicalDeductionFiveObjects10BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT4 837 |
| ✓ | BbhLogicalDeductionFiveObjects11BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT1 239 |
| ✓ | BbhLogicalDeductionFiveObjects12BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT996 |
| ✓ | BbhLogicalDeductionFiveObjects13BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT949 |
| ✓ | BbhLogicalDeductionFiveObjects14BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT485 |
| ✓ | BbhLogicalDeductionFiveObjects15BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT542 |
| ✓ | BbhLogicalDeductionFiveObjects16BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT948 |
| ✓ | BbhLogicalDeductionFiveObjects17BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT591 |
| ✓ | BbhLogicalDeductionFiveObjects18BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT582 |
| ✓ | BbhLogicalDeductionFiveObjects19BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT974 |
| ✓ | BbhLogicalDeductionFiveObjects20BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 406 |
| ✓ | BbhLogicalDeductionFiveObjects21BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 486 |
| ✓ | BbhLogicalDeductionFiveObjects22BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT447 |
| ✓ | BbhLogicalDeductionFiveObjects23BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 304 |
| ✓ | BbhLogicalDeductionFiveObjects24BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT635 |
| ✓ | BbhLogicalDeductionFiveObjects25BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT917 |
| ✓ | BbhLogicalDeductionFiveObjects26BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT441 |
| ✓ | BbhLogicalDeductionFiveObjects27BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT918 |
| ✓ | BbhLogicalDeductionFiveObjects28BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT939 |
| ✓ | BbhLogicalDeductionFiveObjects29BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT516 |
| ✓ | BbhLogicalDeductionFiveObjects30BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT424 |
| ✓ | BbhLogicalDeductionFiveObjects8BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT856 |
| ✓ | BbhLogicalDeductionFiveObjects9BOorder five objects from a set of ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT766 |
Reasoning - BBH logical deduction - order 5 objects (set 2)
32 passed |
100.0% |
IN34 949 OUT27 255 | |
| ✓ | BbhLogicalDeductionFive189BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT642 |
| ✓ | BbhLogicalDeductionFive190BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 023 |
| ✓ | BbhLogicalDeductionFive191BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT496 |
| ✓ | BbhLogicalDeductionFive192BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 115 |
| ✓ | BbhLogicalDeductionFive193BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT823 |
| ✓ | BbhLogicalDeductionFive194BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT481 |
| ✓ | BbhLogicalDeductionFive195BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT696 |
| ✓ | BbhLogicalDeductionFive196BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT506 |
| ✓ | BbhLogicalDeductionFive197BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT654 |
| ✓ | BbhLogicalDeductionFive198BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT936 |
| ✓ | BbhLogicalDeductionFive199BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT2 702 |
| ✓ | BbhLogicalDeductionFive200BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT393 |
| ✓ | BbhLogicalDeductionFive201BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT988 |
| ✓ | BbhLogicalDeductionFive202BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT1 400 |
| ✓ | BbhLogicalDeductionFive203BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT996 |
| ✓ | BbhLogicalDeductionFive204BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT513 |
| ✓ | BbhLogicalDeductionFive205BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT609 |
| ✓ | BbhLogicalDeductionFive206BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT813 |
| ✓ | BbhLogicalDeductionFive207BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT618 |
| ✓ | BbhLogicalDeductionFive208BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 355 |
| ✓ | BbhLogicalDeductionFive209BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT693 |
| ✓ | BbhLogicalDeductionFive210BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT792 |
| ✓ | BbhLogicalDeductionFive211BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT1 557 |
| ✓ | BbhLogicalDeductionFive212BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT401 |
| ✓ | BbhLogicalDeductionFive213BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT1 059 |
| ✓ | BbhLogicalDeductionFive214BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT436 |
| ✓ | BbhLogicalDeductionFive215BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT1 021 |
| ✓ | BbhLogicalDeductionFive216BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT881 |
| ✓ | BbhLogicalDeductionFive217BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT835 |
| ✓ | BbhLogicalDeductionFive218BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT516 |
| ✓ | BbhLogicalDeductionFive219BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT834 |
| ✓ | BbhLogicalDeductionFive220BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT471 |
Reasoning - BBH logical deduction - order 5 objects (set 3)
29 passed |
100.0% |
IN31 763 OUT20 023 | |
| ✓ | BbhLogicalDeductionFive221BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 202 |
| ✓ | BbhLogicalDeductionFive222BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT690 |
| ✓ | BbhLogicalDeductionFive223BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT1 104 |
| ✓ | BbhLogicalDeductionFive224BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT557 |
| ✓ | BbhLogicalDeductionFive225BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT562 |
| ✓ | BbhLogicalDeductionFive226BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT742 |
| ✓ | BbhLogicalDeductionFive227BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 230 |
| ✓ | BbhLogicalDeductionFive228BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT493 |
| ✓ | BbhLogicalDeductionFive229BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT511 |
| ✓ | BbhLogicalDeductionFive230BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT482 |
| ✓ | BbhLogicalDeductionFive231BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT1 062 |
| ✓ | BbhLogicalDeductionFive232BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT452 |
| ✓ | BbhLogicalDeductionFive233BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 608 |
| ✓ | BbhLogicalDeductionFive234BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT526 |
| ✓ | BbhLogicalDeductionFive235BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT526 |
| ✓ | BbhLogicalDeductionFive236BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT700 |
| ✓ | BbhLogicalDeductionFive237BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT570 |
| ✓ | BbhLogicalDeductionFive238BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT578 |
| ✓ | BbhLogicalDeductionFive239BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT868 |
| ✓ | BbhLogicalDeductionFive240BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT454 |
| ✓ | BbhLogicalDeductionFive241BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT778 |
| ✓ | BbhLogicalDeductionFive242BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT510 |
| ✓ | BbhLogicalDeductionFive243BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT683 |
| ✓ | BbhLogicalDeductionFive244BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT514 |
| ✓ | BbhLogicalDeductionFive245BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT597 |
| ✓ | BbhLogicalDeductionFive246BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 121 OUT716 |
| ✓ | BbhLogicalDeductionFive247BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT503 |
| ✓ | BbhLogicalDeductionFive248BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT385 |
| ✓ | BbhLogicalDeductionFive249BQorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT420 |
Reasoning - BBH logical deduction - order 5 objects (set 4)
32 passed |
100.0% |
IN35 110 OUT24 919 | |
| ✓ | BbhLogicalDeductionFiveObjects31BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT390 |
| ✓ | BbhLogicalDeductionFiveObjects32BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT757 |
| ✓ | BbhLogicalDeductionFiveObjects33BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT573 |
| ✓ | BbhLogicalDeductionFiveObjects34BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT763 |
| ✓ | BbhLogicalDeductionFiveObjects35BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT675 |
| ✓ | BbhLogicalDeductionFiveObjects36BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT547 |
| ✓ | BbhLogicalDeductionFiveObjects37BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT966 |
| ✓ | BbhLogicalDeductionFiveObjects38BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT874 |
| ✓ | BbhLogicalDeductionFiveObjects39BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT432 |
| ✓ | BbhLogicalDeductionFiveObjects40BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT1 193 |
| ✓ | BbhLogicalDeductionFiveObjects41BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 593 |
| ✓ | BbhLogicalDeductionFiveObjects42BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT527 |
| ✓ | BbhLogicalDeductionFiveObjects43BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT564 |
| ✓ | BbhLogicalDeductionFiveObjects44BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 157 |
| ✓ | BbhLogicalDeductionFiveObjects45BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT819 |
| ✓ | BbhLogicalDeductionFiveObjects46BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT732 |
| ✓ | BbhLogicalDeductionFiveObjects47BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 420 |
| ✓ | BbhLogicalDeductionFiveObjects48BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT840 |
| ✓ | BbhLogicalDeductionFiveObjects49BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT675 |
| ✓ | BbhLogicalDeductionFiveObjects50BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT807 |
| ✓ | BbhLogicalDeductionFiveObjects51BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT548 |
| ✓ | BbhLogicalDeductionFiveObjects52BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT502 |
| ✓ | BbhLogicalDeductionFiveObjects53BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT947 |
| ✓ | BbhLogicalDeductionFiveObjects54BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT642 |
| ✓ | BbhLogicalDeductionFiveObjects55BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT657 |
| ✓ | BbhLogicalDeductionFiveObjects56BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT529 |
| ✓ | BbhLogicalDeductionFiveObjects57BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT750 |
| ✓ | BbhLogicalDeductionFiveObjects58BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT704 |
| ✓ | BbhLogicalDeductionFiveObjects59BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT1 503 |
| ✓ | BbhLogicalDeductionFiveObjects60BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT540 |
| ✓ | BbhLogicalDeductionFiveObjects61BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT581 |
| ✓ | BbhLogicalDeductionFiveObjects62BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT712 |
Reasoning - BBH logical deduction - order 5 objects (set 5)
32 passed |
100.0% |
IN35 118 OUT24 970 | |
| ✓ | BbhLogicalDeductionFiveObjects63BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT588 |
| ✓ | BbhLogicalDeductionFiveObjects64BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 391 |
| ✓ | BbhLogicalDeductionFiveObjects65BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT889 |
| ✓ | BbhLogicalDeductionFiveObjects66BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT877 |
| ✓ | BbhLogicalDeductionFiveObjects67BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT1 029 |
| ✓ | BbhLogicalDeductionFiveObjects68BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT653 |
| ✓ | BbhLogicalDeductionFiveObjects69BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT650 |
| ✓ | BbhLogicalDeductionFiveObjects70BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT788 |
| ✓ | BbhLogicalDeductionFiveObjects71BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT740 |
| ✓ | BbhLogicalDeductionFiveObjects72BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT974 |
| ✓ | BbhLogicalDeductionFiveObjects73BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT454 |
| ✓ | BbhLogicalDeductionFiveObjects74BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT607 |
| ✓ | BbhLogicalDeductionFiveObjects75BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT464 |
| ✓ | BbhLogicalDeductionFiveObjects76BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT615 |
| ✓ | BbhLogicalDeductionFiveObjects77BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT1 361 |
| ✓ | BbhLogicalDeductionFiveObjects78BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT677 |
| ✓ | BbhLogicalDeductionFiveObjects79BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT407 |
| ✓ | BbhLogicalDeductionFiveObjects80BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT723 |
| ✓ | BbhLogicalDeductionFiveObjects81BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT572 |
| ✓ | BbhLogicalDeductionFiveObjects82BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT2 611 |
| ✓ | BbhLogicalDeductionFiveObjects83BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT429 |
| ✓ | BbhLogicalDeductionFiveObjects84BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT655 |
| ✓ | BbhLogicalDeductionFiveObjects85BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT817 |
| ✓ | BbhLogicalDeductionFiveObjects86BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT326 |
| ✓ | BbhLogicalDeductionFiveObjects87BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT532 |
| ✓ | BbhLogicalDeductionFiveObjects88BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT694 |
| ✓ | BbhLogicalDeductionFiveObjects89BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT839 |
| ✓ | BbhLogicalDeductionFiveObjects90BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT994 |
| ✓ | BbhLogicalDeductionFiveObjects91BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT698 |
| ✓ | BbhLogicalDeductionFiveObjects92BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT536 |
| ✓ | BbhLogicalDeductionFiveObjects93BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT802 |
| ✓ | BbhLogicalDeductionFiveObjects94BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT578 |
Reasoning - BBH logical deduction - order 5 objects (set 6)
32 passed |
100.0% |
IN35 052 OUT23 664 | |
| ✓ | BbhLogicalDeductionFiveObjects100BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT351 |
| ✓ | BbhLogicalDeductionFiveObjects101BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT797 |
| ✓ | BbhLogicalDeductionFiveObjects102BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT474 |
| ✓ | BbhLogicalDeductionFiveObjects103BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT1 032 |
| ✓ | BbhLogicalDeductionFiveObjects104BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT499 |
| ✓ | BbhLogicalDeductionFiveObjects105BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT1 933 |
| ✓ | BbhLogicalDeductionFiveObjects106BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT538 |
| ✓ | BbhLogicalDeductionFiveObjects107BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT740 |
| ✓ | BbhLogicalDeductionFiveObjects108BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT668 |
| ✓ | BbhLogicalDeductionFiveObjects109BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT748 |
| ✓ | BbhLogicalDeductionFiveObjects110BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT771 |
| ✓ | BbhLogicalDeductionFiveObjects111BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT950 |
| ✓ | BbhLogicalDeductionFiveObjects112BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT932 |
| ✓ | BbhLogicalDeductionFiveObjects113BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT499 |
| ✓ | BbhLogicalDeductionFiveObjects114BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT448 |
| ✓ | BbhLogicalDeductionFiveObjects115BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT888 |
| ✓ | BbhLogicalDeductionFiveObjects116BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT1 210 |
| ✓ | BbhLogicalDeductionFiveObjects117BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT610 |
| ✓ | BbhLogicalDeductionFiveObjects118BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT641 |
| ✓ | BbhLogicalDeductionFiveObjects119BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT776 |
| ✓ | BbhLogicalDeductionFiveObjects120BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT748 |
| ✓ | BbhLogicalDeductionFiveObjects121BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT646 |
| ✓ | BbhLogicalDeductionFiveObjects122BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT750 |
| ✓ | BbhLogicalDeductionFiveObjects123BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT610 |
| ✓ | BbhLogicalDeductionFiveObjects124BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT425 |
| ✓ | BbhLogicalDeductionFiveObjects125BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT579 |
| ✓ | BbhLogicalDeductionFiveObjects126BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT939 |
| ✓ | BbhLogicalDeductionFiveObjects95BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT637 |
| ✓ | BbhLogicalDeductionFiveObjects96BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT544 |
| ✓ | BbhLogicalDeductionFiveObjects97BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT1 191 |
| ✓ | BbhLogicalDeductionFiveObjects98BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT432 |
| ✓ | BbhLogicalDeductionFiveObjects99BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT658 |
Reasoning - BBH logical deduction - order 5 objects (set 7)
32 passed |
100.0% |
IN35 085 OUT27 755 | |
| ✓ | BbhLogicalDeductionFiveObjects127BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT714 |
| ✓ | BbhLogicalDeductionFiveObjects128BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT966 |
| ✓ | BbhLogicalDeductionFiveObjects129BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT1 883 |
| ✓ | BbhLogicalDeductionFiveObjects130BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 531 |
| ✓ | BbhLogicalDeductionFiveObjects131BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT1 280 |
| ✓ | BbhLogicalDeductionFiveObjects132BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT704 |
| ✓ | BbhLogicalDeductionFiveObjects133BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT633 |
| ✓ | BbhLogicalDeductionFiveObjects134BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT584 |
| ✓ | BbhLogicalDeductionFiveObjects135BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT622 |
| ✓ | BbhLogicalDeductionFiveObjects136BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT1 304 |
| ✓ | BbhLogicalDeductionFiveObjects137BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT637 |
| ✓ | BbhLogicalDeductionFiveObjects138BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT1 027 |
| ✓ | BbhLogicalDeductionFiveObjects139BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT925 |
| ✓ | BbhLogicalDeductionFiveObjects140BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT881 |
| ✓ | BbhLogicalDeductionFiveObjects141BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT1 108 |
| ✓ | BbhLogicalDeductionFiveObjects142BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT723 |
| ✓ | BbhLogicalDeductionFiveObjects143BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT532 |
| ✓ | BbhLogicalDeductionFiveObjects144BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT1 457 |
| ✓ | BbhLogicalDeductionFiveObjects145BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT872 |
| ✓ | BbhLogicalDeductionFiveObjects146BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT355 |
| ✓ | BbhLogicalDeductionFiveObjects147BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT879 |
| ✓ | BbhLogicalDeductionFiveObjects148BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT399 |
| ✓ | BbhLogicalDeductionFiveObjects149BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT942 |
| ✓ | BbhLogicalDeductionFiveObjects150BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT658 |
| ✓ | BbhLogicalDeductionFiveObjects151BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT424 |
| ✓ | BbhLogicalDeductionFiveObjects152BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT734 |
| ✓ | BbhLogicalDeductionFiveObjects153BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT880 |
| ✓ | BbhLogicalDeductionFiveObjects154BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT641 |
| ✓ | BbhLogicalDeductionFiveObjects155BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT1 291 |
| ✓ | BbhLogicalDeductionFiveObjects156BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT912 |
| ✓ | BbhLogicalDeductionFiveObjects157BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT600 |
| ✓ | BbhLogicalDeductionFiveObjects158BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT657 |
Reasoning - BBH logical deduction - order 5 objects (set 8)
30 passed |
100.0% |
IN32 775 OUT25 229 | |
| ✓ | BbhLogicalDeductionFiveObjects159BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT628 |
| ✓ | BbhLogicalDeductionFiveObjects160BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT690 |
| ✓ | BbhLogicalDeductionFiveObjects161BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT616 |
| ✓ | BbhLogicalDeductionFiveObjects162BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT698 |
| ✓ | BbhLogicalDeductionFiveObjects163BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT530 |
| ✓ | BbhLogicalDeductionFiveObjects164BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT662 |
| ✓ | BbhLogicalDeductionFiveObjects165BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT1 164 |
| ✓ | BbhLogicalDeductionFiveObjects166BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT431 |
| ✓ | BbhLogicalDeductionFiveObjects167BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT771 |
| ✓ | BbhLogicalDeductionFiveObjects168BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT1 512 |
| ✓ | BbhLogicalDeductionFiveObjects169BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT805 |
| ✓ | BbhLogicalDeductionFiveObjects170BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT820 |
| ✓ | BbhLogicalDeductionFiveObjects171BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 060 OUT802 |
| ✓ | BbhLogicalDeductionFiveObjects172BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 231 |
| ✓ | BbhLogicalDeductionFiveObjects173BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT668 |
| ✓ | BbhLogicalDeductionFiveObjects174BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT601 |
| ✓ | BbhLogicalDeductionFiveObjects175BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT992 |
| ✓ | BbhLogicalDeductionFiveObjects176BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT1 178 |
| ✓ | BbhLogicalDeductionFiveObjects177BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT834 |
| ✓ | BbhLogicalDeductionFiveObjects178BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT490 |
| ✓ | BbhLogicalDeductionFiveObjects179BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT754 |
| ✓ | BbhLogicalDeductionFiveObjects180BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 058 OUT1 382 |
| ✓ | BbhLogicalDeductionFiveObjects181BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT659 |
| ✓ | BbhLogicalDeductionFiveObjects182BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT624 |
| ✓ | BbhLogicalDeductionFiveObjects183BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT463 |
| ✓ | BbhLogicalDeductionFiveObjects184BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT2 523 |
| ✓ | BbhLogicalDeductionFiveObjects185BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT643 |
| ✓ | BbhLogicalDeductionFiveObjects186BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT621 |
| ✓ | BbhLogicalDeductionFiveObjects187BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT776 |
| ✓ | BbhLogicalDeductionFiveObjects188BPorder 5 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT661 |
Reasoning - BBH logical deduction - order 7 objects (set 1)
33 passed |
100.0% |
IN37 746 OUT40 308 | |
| ✓ | BbhLogicalDeductionSeven10BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT2 051 |
| ✓ | BbhLogicalDeductionSeven11BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT902 |
| ✓ | BbhLogicalDeductionSeven12BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT591 |
| ✓ | BbhLogicalDeductionSeven13BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT803 |
| ✓ | BbhLogicalDeductionSeven14BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 385 |
| ✓ | BbhLogicalDeductionSeven15BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT922 |
| ✓ | BbhLogicalDeductionSeven16BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT3 742 |
| ✓ | BbhLogicalDeductionSeven17BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT927 |
| ✓ | BbhLogicalDeductionSeven18BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT1 297 |
| ✓ | BbhLogicalDeductionSeven19BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT772 |
| ✓ | BbhLogicalDeductionSeven20BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT905 |
| ✓ | BbhLogicalDeductionSeven21BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT1 023 |
| ✓ | BbhLogicalDeductionSeven22BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT1 163 |
| ✓ | BbhLogicalDeductionSeven23BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT1 131 |
| ✓ | BbhLogicalDeductionSeven24BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 156 |
| ✓ | BbhLogicalDeductionSeven25BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT855 |
| ✓ | BbhLogicalDeductionSeven26BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 290 |
| ✓ | BbhLogicalDeductionSeven27BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT868 |
| ✓ | BbhLogicalDeductionSeven28BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 484 |
| ✓ | BbhLogicalDeductionSeven29BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 535 |
| ✓ | BbhLogicalDeductionSeven30BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT1 319 |
| ✓ | BbhLogicalDeductionSeven31BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 084 OUT1 280 |
| ✓ | BbhLogicalDeductionSeven32BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT988 |
| ✓ | BbhLogicalDeductionSeven33BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT1 499 |
| ✓ | BbhLogicalDeductionSeven34BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT2 327 |
| ✓ | BbhLogicalDeductionSeven35BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT994 |
| ✓ | BbhLogicalDeductionSeven36BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT830 |
| ✓ | BbhLogicalDeductionSeven37BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT800 |
| ✓ | BbhLogicalDeductionSeven38BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 134 OUT758 |
| ✓ | BbhLogicalDeductionSeven39BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 100 OUT785 |
| ✓ | BbhLogicalDeductionSeven40BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 310 |
| ✓ | BbhLogicalDeductionSeven8BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 136 OUT822 |
| ✓ | BbhLogicalDeductionSeven9BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT794 |
Reasoning - BBH logical deduction - order 7 objects (set 2)
33 passed |
100.0% |
IN37 433 OUT48 544 | |
| ✓ | BbhLogicalDeductionSeven41BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT825 |
| ✓ | BbhLogicalDeductionSeven42BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 135 OUT2 160 |
| ✓ | BbhLogicalDeductionSeven43BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 145 OUT994 |
| ✓ | BbhLogicalDeductionSeven44BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 133 OUT7 324 |
| ✓ | BbhLogicalDeductionSeven45BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT823 |
| ✓ | BbhLogicalDeductionSeven46BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 082 OUT1 004 |
| ✓ | BbhLogicalDeductionSeven47BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 236 |
| ✓ | BbhLogicalDeductionSeven48BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT875 |
| ✓ | BbhLogicalDeductionSeven49BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 139 OUT1 325 |
| ✓ | BbhLogicalDeductionSeven50BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 153 |
| ✓ | BbhLogicalDeductionSeven51BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT1 170 |
| ✓ | BbhLogicalDeductionSeven52BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 093 OUT1 707 |
| ✓ | BbhLogicalDeductionSeven53BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT1 154 |
| ✓ | BbhLogicalDeductionSeven54BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT855 |
| ✓ | BbhLogicalDeductionSeven55BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT810 |
| ✓ | BbhLogicalDeductionSeven56BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT2 957 |
| ✓ | BbhLogicalDeductionSeven57BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 008 |
| ✓ | BbhLogicalDeductionSeven58BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 310 |
| ✓ | BbhLogicalDeductionSeven59BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT1 634 |
| ✓ | BbhLogicalDeductionSeven60BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 093 OUT782 |
| ✓ | BbhLogicalDeductionSeven61BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT922 |
| ✓ | BbhLogicalDeductionSeven62BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 075 |
| ✓ | BbhLogicalDeductionSeven63BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 379 |
| ✓ | BbhLogicalDeductionSeven64BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 084 OUT1 133 |
| ✓ | BbhLogicalDeductionSeven65BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 519 |
| ✓ | BbhLogicalDeductionSeven66BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT924 |
| ✓ | BbhLogicalDeductionSeven67BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT1 016 |
| ✓ | BbhLogicalDeductionSeven68BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 139 OUT1 595 |
| ✓ | BbhLogicalDeductionSeven69BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT809 |
| ✓ | BbhLogicalDeductionSeven70BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT2 486 |
| ✓ | BbhLogicalDeductionSeven71BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT930 |
| ✓ | BbhLogicalDeductionSeven72BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 135 OUT779 |
| ✓ | BbhLogicalDeductionSeven73BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT1 871 |
Reasoning - BBH logical deduction - order 7 objects (set 3)
31 passed |
100.0% |
IN35 412 OUT41 138 | |
| ✓ | BbhLogicalDeductionSeven100BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 139 OUT1 455 |
| ✓ | BbhLogicalDeductionSeven101BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT997 |
| ✓ | BbhLogicalDeductionSeven102BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT1 043 |
| ✓ | BbhLogicalDeductionSeven103BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT566 |
| ✓ | BbhLogicalDeductionSeven104BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT1 219 |
| ✓ | BbhLogicalDeductionSeven74BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT962 |
| ✓ | BbhLogicalDeductionSeven75BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT941 |
| ✓ | BbhLogicalDeductionSeven76BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT2 393 |
| ✓ | BbhLogicalDeductionSeven77BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT990 |
| ✓ | BbhLogicalDeductionSeven78BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT2 313 |
| ✓ | BbhLogicalDeductionSeven79BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT945 |
| ✓ | BbhLogicalDeductionSeven80BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 082 OUT742 |
| ✓ | BbhLogicalDeductionSeven81BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 183 OUT1 217 |
| ✓ | BbhLogicalDeductionSeven82BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 352 |
| ✓ | BbhLogicalDeductionSeven83BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 138 OUT1 090 |
| ✓ | BbhLogicalDeductionSeven84BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 082 OUT719 |
| ✓ | BbhLogicalDeductionSeven85BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT1 064 |
| ✓ | BbhLogicalDeductionSeven86BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 694 |
| ✓ | BbhLogicalDeductionSeven87BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT909 |
| ✓ | BbhLogicalDeductionSeven88BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 163 OUT1 105 |
| ✓ | BbhLogicalDeductionSeven89BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 135 OUT738 |
| ✓ | BbhLogicalDeductionSeven90BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT1 172 |
| ✓ | BbhLogicalDeductionSeven91BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 014 |
| ✓ | BbhLogicalDeductionSeven92BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 174 |
| ✓ | BbhLogicalDeductionSeven93BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT1 216 |
| ✓ | BbhLogicalDeductionSeven94BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 483 |
| ✓ | BbhLogicalDeductionSeven95BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 855 |
| ✓ | BbhLogicalDeductionSeven96BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT2 677 |
| ✓ | BbhLogicalDeductionSeven97BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT2 569 |
| ✓ | BbhLogicalDeductionSeven98BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT932 |
| ✓ | BbhLogicalDeductionSeven99BQorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 154 OUT2 592 |
Reasoning - BBH logical deduction - order 7 objects (set 4)
32 passed |
100.0% |
IN36 356 OUT43 347 | |
| ✓ | BbhLogicalDeductionSeven105BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT1 052 |
| ✓ | BbhLogicalDeductionSeven106BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT1 263 |
| ✓ | BbhLogicalDeductionSeven107BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT933 |
| ✓ | BbhLogicalDeductionSeven108BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT1 617 |
| ✓ | BbhLogicalDeductionSeven109BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT2 119 |
| ✓ | BbhLogicalDeductionSeven110BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT1 507 |
| ✓ | BbhLogicalDeductionSeven111BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT1 020 |
| ✓ | BbhLogicalDeductionSeven112BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 101 OUT1 306 |
| ✓ | BbhLogicalDeductionSeven113BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT754 |
| ✓ | BbhLogicalDeductionSeven114BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT1 105 |
| ✓ | BbhLogicalDeductionSeven115BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT914 |
| ✓ | BbhLogicalDeductionSeven116BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 710 |
| ✓ | BbhLogicalDeductionSeven117BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT2 731 |
| ✓ | BbhLogicalDeductionSeven118BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT982 |
| ✓ | BbhLogicalDeductionSeven119BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 180 |
| ✓ | BbhLogicalDeductionSeven120BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 165 OUT1 497 |
| ✓ | BbhLogicalDeductionSeven121BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 098 OUT1 478 |
| ✓ | BbhLogicalDeductionSeven122BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT2 006 |
| ✓ | BbhLogicalDeductionSeven123BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 094 OUT1 033 |
| ✓ | BbhLogicalDeductionSeven124BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT980 |
| ✓ | BbhLogicalDeductionSeven125BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT1 380 |
| ✓ | BbhLogicalDeductionSeven126BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT816 |
| ✓ | BbhLogicalDeductionSeven127BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT1 095 |
| ✓ | BbhLogicalDeductionSeven128BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 689 |
| ✓ | BbhLogicalDeductionSeven129BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT2 191 |
| ✓ | BbhLogicalDeductionSeven130BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 090 OUT1 625 |
| ✓ | BbhLogicalDeductionSeven131BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 485 |
| ✓ | BbhLogicalDeductionSeven132BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 177 |
| ✓ | BbhLogicalDeductionSeven133BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 143 |
| ✓ | BbhLogicalDeductionSeven134BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT687 |
| ✓ | BbhLogicalDeductionSeven135BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 136 OUT1 041 |
| ✓ | BbhLogicalDeductionSeven136BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 081 OUT1 831 |
Reasoning - BBH logical deduction - order 7 objects (set 5)
32 passed |
100.0% |
IN36 467 OUT39 007 | |
| ✓ | BbhLogicalDeductionSeven137BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT652 |
| ✓ | BbhLogicalDeductionSeven138BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT968 |
| ✓ | BbhLogicalDeductionSeven139BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT594 |
| ✓ | BbhLogicalDeductionSeven140BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT983 |
| ✓ | BbhLogicalDeductionSeven141BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT855 |
| ✓ | BbhLogicalDeductionSeven142BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 057 |
| ✓ | BbhLogicalDeductionSeven143BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT827 |
| ✓ | BbhLogicalDeductionSeven144BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT1 263 |
| ✓ | BbhLogicalDeductionSeven145BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT1 589 |
| ✓ | BbhLogicalDeductionSeven146BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT1 908 |
| ✓ | BbhLogicalDeductionSeven147BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT1 028 |
| ✓ | BbhLogicalDeductionSeven148BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT944 |
| ✓ | BbhLogicalDeductionSeven149BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT1 542 |
| ✓ | BbhLogicalDeductionSeven150BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 064 |
| ✓ | BbhLogicalDeductionSeven151BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 718 |
| ✓ | BbhLogicalDeductionSeven152BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 145 OUT919 |
| ✓ | BbhLogicalDeductionSeven153BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 094 OUT2 394 |
| ✓ | BbhLogicalDeductionSeven154BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT891 |
| ✓ | BbhLogicalDeductionSeven155BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT981 |
| ✓ | BbhLogicalDeductionSeven156BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT801 |
| ✓ | BbhLogicalDeductionSeven157BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 084 OUT1 659 |
| ✓ | BbhLogicalDeductionSeven158BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT1 123 |
| ✓ | BbhLogicalDeductionSeven159BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT991 |
| ✓ | BbhLogicalDeductionSeven160BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 021 |
| ✓ | BbhLogicalDeductionSeven161BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 090 OUT3 070 |
| ✓ | BbhLogicalDeductionSeven162BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 090 |
| ✓ | BbhLogicalDeductionSeven163BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT953 |
| ✓ | BbhLogicalDeductionSeven164BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 208 |
| ✓ | BbhLogicalDeductionSeven165BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT975 |
| ✓ | BbhLogicalDeductionSeven166BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 793 |
| ✓ | BbhLogicalDeductionSeven167BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 091 OUT920 |
| ✓ | BbhLogicalDeductionSeven168BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 226 |
Reasoning - BBH logical deduction - order 7 objects (set 6)
32 passed |
100.0% |
IN36 304 OUT37 234 | |
| ✓ | BbhLogicalDeductionSeven169BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 372 |
| ✓ | BbhLogicalDeductionSeven170BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT877 |
| ✓ | BbhLogicalDeductionSeven171BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT608 |
| ✓ | BbhLogicalDeductionSeven172BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 087 OUT1 630 |
| ✓ | BbhLogicalDeductionSeven173BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT1 210 |
| ✓ | BbhLogicalDeductionSeven174BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 552 |
| ✓ | BbhLogicalDeductionSeven175BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT997 |
| ✓ | BbhLogicalDeductionSeven176BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT1 398 |
| ✓ | BbhLogicalDeductionSeven177BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 072 |
| ✓ | BbhLogicalDeductionSeven178BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 079 OUT919 |
| ✓ | BbhLogicalDeductionSeven179BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT677 |
| ✓ | BbhLogicalDeductionSeven180BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 518 |
| ✓ | BbhLogicalDeductionSeven181BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 095 OUT1 428 |
| ✓ | BbhLogicalDeductionSeven182BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT735 |
| ✓ | BbhLogicalDeductionSeven183BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 077 OUT916 |
| ✓ | BbhLogicalDeductionSeven184BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT2 235 |
| ✓ | BbhLogicalDeductionSeven185BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 687 |
| ✓ | BbhLogicalDeductionSeven186BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 097 OUT1 083 |
| ✓ | BbhLogicalDeductionSeven187BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT1 809 |
| ✓ | BbhLogicalDeductionSeven188BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 097 OUT911 |
| ✓ | BbhLogicalDeductionSeven189BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 177 OUT913 |
| ✓ | BbhLogicalDeductionSeven190BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT930 |
| ✓ | BbhLogicalDeductionSeven191BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT1 091 |
| ✓ | BbhLogicalDeductionSeven192BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT743 |
| ✓ | BbhLogicalDeductionSeven193BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT1 145 |
| ✓ | BbhLogicalDeductionSeven194BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 077 |
| ✓ | BbhLogicalDeductionSeven195BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT1 208 |
| ✓ | BbhLogicalDeductionSeven196BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT996 |
| ✓ | BbhLogicalDeductionSeven197BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 134 OUT1 028 |
| ✓ | BbhLogicalDeductionSeven198BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT1 489 |
| ✓ | BbhLogicalDeductionSeven199BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT735 |
| ✓ | BbhLogicalDeductionSeven200BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 124 OUT1 245 |
Reasoning - BBH logical deduction - order 7 objects (set 7)
32 passed |
100.0% |
IN36 429 OUT38 014 | |
| ✓ | BbhLogicalDeductionSeven201BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 143 OUT957 |
| ✓ | BbhLogicalDeductionSeven202BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 066 |
| ✓ | BbhLogicalDeductionSeven203BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 144 OUT841 |
| ✓ | BbhLogicalDeductionSeven204BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 026 |
| ✓ | BbhLogicalDeductionSeven205BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT880 |
| ✓ | BbhLogicalDeductionSeven206BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 140 OUT946 |
| ✓ | BbhLogicalDeductionSeven207BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 616 |
| ✓ | BbhLogicalDeductionSeven208BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 093 OUT710 |
| ✓ | BbhLogicalDeductionSeven209BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 094 OUT1 624 |
| ✓ | BbhLogicalDeductionSeven210BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT2 050 |
| ✓ | BbhLogicalDeductionSeven211BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 613 |
| ✓ | BbhLogicalDeductionSeven212BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT1 213 |
| ✓ | BbhLogicalDeductionSeven213BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 045 |
| ✓ | BbhLogicalDeductionSeven214BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 082 OUT814 |
| ✓ | BbhLogicalDeductionSeven215BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 139 OUT910 |
| ✓ | BbhLogicalDeductionSeven216BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 147 OUT1 009 |
| ✓ | BbhLogicalDeductionSeven217BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 124 OUT1 389 |
| ✓ | BbhLogicalDeductionSeven218BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 139 OUT1 131 |
| ✓ | BbhLogicalDeductionSeven219BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 146 OUT846 |
| ✓ | BbhLogicalDeductionSeven220BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT714 |
| ✓ | BbhLogicalDeductionSeven221BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 090 OUT1 264 |
| ✓ | BbhLogicalDeductionSeven222BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 139 OUT2 784 |
| ✓ | BbhLogicalDeductionSeven223BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT1 044 |
| ✓ | BbhLogicalDeductionSeven224BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 097 OUT1 034 |
| ✓ | BbhLogicalDeductionSeven225BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 183 OUT1 434 |
| ✓ | BbhLogicalDeductionSeven226BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 083 OUT702 |
| ✓ | BbhLogicalDeductionSeven227BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT984 |
| ✓ | BbhLogicalDeductionSeven228BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT1 599 |
| ✓ | BbhLogicalDeductionSeven229BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 342 |
| ✓ | BbhLogicalDeductionSeven230BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 142 OUT1 384 |
| ✓ | BbhLogicalDeductionSeven231BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 161 OUT1 016 |
| ✓ | BbhLogicalDeductionSeven232BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 027 |
Reasoning - BBH logical deduction - order 7 objects (set 8)
30 passed |
100.0% |
IN33 188 OUT23 189 | |
| ✓ | BbhLogicalDeductionSeven233BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 148 OUT1 366 |
| ✓ | BbhLogicalDeductionSeven234BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 134 OUT1 121 |
| ✓ | BbhLogicalDeductionSeven235BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 080 OUT687 |
| ✓ | BbhLogicalDeductionSeven236BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT1 114 |
| ✓ | BbhLogicalDeductionSeven237BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT1 204 |
| ✓ | BbhLogicalDeductionSeven238BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT1 079 |
| ✓ | BbhLogicalDeductionSeven239BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT1 186 |
| ✓ | BbhLogicalDeductionSeven240BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 174 |
| ✓ | BbhLogicalDeductionSeven241BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 145 OUT833 |
| ✓ | BbhLogicalDeductionSeven242BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 177 OUT1 203 |
| ✓ | BbhLogicalDeductionSeven243BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT961 |
| ✓ | BbhLogicalDeductionSeven244BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT1 261 |
| ✓ | BbhLogicalDeductionSeven245BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 097 OUT873 |
| ✓ | BbhLogicalDeductionSeven246BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 152 OUT865 |
| ✓ | BbhLogicalDeductionSeven247BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT657 |
| ✓ | BbhLogicalDeductionSeven248BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT974 |
| ✓ | BbhLogicalDeductionSeven249BRorder 7 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 138 OUT1 691 |
| ✓ | BbhLogicalDeductionThree10BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 059 OUT340 |
| ✓ | BbhLogicalDeductionThree11BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT330 |
| ✓ | BbhLogicalDeductionThree12BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT357 |
| ✓ | BbhLogicalDeductionThree13BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 054 OUT358 |
| ✓ | BbhLogicalDeductionThree14BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT662 |
| ✓ | BbhLogicalDeductionThree15BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT389 |
| ✓ | BbhLogicalDeductionThree16BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 053 OUT447 |
| ✓ | BbhLogicalDeductionThree17BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT388 |
| ✓ | BbhLogicalDeductionThree18BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT234 |
| ✓ | BbhLogicalDeductionThree19BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT325 |
| ✓ | BbhLogicalDeductionThree20BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT373 |
| ✓ | BbhLogicalDeductionThree8BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT402 |
| ✓ | BbhLogicalDeductionThree9BRorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT335 |
Reasoning - BBH movie recommendation (set 1)
30 passed2 failed |
93.8% |
IN32 573 OUT27 707 | |
| ✓ | BbhLogicalDeductionThreeObjects243BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT373 |
| ✓ | BbhLogicalDeductionThreeObjects244BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT370 |
| ✓ | BbhLogicalDeductionThreeObjects245BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 057 OUT442 |
| ✓ | BbhLogicalDeductionThreeObjects246BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT450 |
| ✓ | BbhLogicalDeductionThreeObjects247BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT336 |
| ✓ | BbhLogicalDeductionThreeObjects248BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT443 |
| ✓ | BbhLogicalDeductionThreeObjects249BTorder 3 objects from ordering constraints1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT306 |
| ✓ | BbhMovieRecommendation10BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT2 063 |
| ✓ | BbhMovieRecommendation11BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT635 |
| ✓ | BbhMovieRecommendation12BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT681 |
| ✓ | BbhMovieRecommendation13BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT645 |
| ✓ | BbhMovieRecommendation14BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT683 |
| ✓ | BbhMovieRecommendation15BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 329 |
| ✓ | BbhMovieRecommendation16BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT702 |
| ✓ | BbhMovieRecommendation17BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT600 |
| ✓ | BbhMovieRecommendation18BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT925 |
| ✓ | BbhMovieRecommendation19BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT617 |
| ✓ | BbhMovieRecommendation20BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 342 |
| ✓ | BbhMovieRecommendation21BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT749 |
| ✓ | BbhMovieRecommendation22BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT744 |
| ✓ | BbhMovieRecommendation23BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT702 |
| ✓ | BbhMovieRecommendation24BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT2 733 |
| × | BbhMovieRecommendation25BTFind a movie similar to Army of Darkness, Go, The Sixth Sense, Alien:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhMovieRecommendation25BT: expected (A), got [(B)] |
Failed | IN1 000 OUT727 |
| ✓ | BbhMovieRecommendation26BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT607 |
| ✓ | BbhMovieRecommendation27BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT775 |
| ✓ | BbhMovieRecommendation28BTFind a movie similar to Toy Story, The Mask, Batman, The Lion King:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 111 |
| ✓ | BbhMovieRecommendation29BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT696 |
| ✓ | BbhMovieRecommendation30BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT658 |
| ✓ | BbhMovieRecommendation31BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 189 |
| ✓ | BbhMovieRecommendation32BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 126 |
| ✓ | BbhMovieRecommendation8BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 959 |
| × | BbhMovieRecommendation9BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(C)")
Context: BbhMovieRecommendation9BT: expected (C), got [(B)] |
Failed | IN1 023 OUT989 |
Reasoning - BBH movie recommendation (set 2)
28 passed4 failed |
87.5% |
IN32 324 OUT37 204 | |
| ✓ | BbhMovieRecommendation33BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 693 |
| ✓ | BbhMovieRecommendation34BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 949 |
| ✓ | BbhMovieRecommendation35BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT1 800 |
| ✓ | BbhMovieRecommendation36BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT860 |
| ✓ | BbhMovieRecommendation37BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT2 171 |
| ✓ | BbhMovieRecommendation38BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT663 |
| × | BbhMovieRecommendation39BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(D)")
Context: BbhMovieRecommendation39BT: expected (D), got [(B)] |
Failed | IN998 OUT2 861 |
| ✓ | BbhMovieRecommendation40BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 689 |
| ✓ | BbhMovieRecommendation41BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT595 |
| ✓ | BbhMovieRecommendation42BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT836 |
| ✓ | BbhMovieRecommendation43BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT883 |
| ✓ | BbhMovieRecommendation44BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT992 |
| ✓ | BbhMovieRecommendation45BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT895 |
| ✓ | BbhMovieRecommendation46BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT195 |
| ✓ | BbhMovieRecommendation47BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT581 |
| ✓ | BbhMovieRecommendation48BTFind a movie similar to Batman, The Mask, The Lion King, Pretty Woman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT602 |
| ✓ | BbhMovieRecommendation49BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT818 |
| ✓ | BbhMovieRecommendation50BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT1 120 |
| ✓ | BbhMovieRecommendation51BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT592 |
| × | BbhMovieRecommendation52BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(A)")
Context: BbhMovieRecommendation52BT: expected (A), got [(B)] |
Failed | IN1 002 OUT675 |
| ✓ | BbhMovieRecommendation53BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT667 |
| ✓ | BbhMovieRecommendation54BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 209 |
| ✓ | BbhMovieRecommendation55BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT681 |
| ✓ | BbhMovieRecommendation56BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT584 |
| ✓ | BbhMovieRecommendation57BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT2 025 |
| ✓ | BbhMovieRecommendation58BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT740 |
| ✓ | BbhMovieRecommendation59BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT753 |
| ✓ | BbhMovieRecommendation60BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT707 |
| ✓ | BbhMovieRecommendation61BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 057 |
| × | BbhMovieRecommendation62BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(B)")
Context: BbhMovieRecommendation62BT: expected (B), got [(C)] |
Failed | IN1 008 OUT3 423 |
| ✓ | BbhMovieRecommendation63BTFind a movie similar to Pretty Woman, Aladdin, The Lion King, Batman:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 084 |
| × | BbhMovieRecommendation64BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("(D)")
Context: BbhMovieRecommendation64BT: expected (D), got [(A)] |
Failed | IN1 009 OUT1 804 |
Reasoning - BBH movie recommendation (set 3)
30 passed |
100.0% |
IN30 348 OUT37 238 | |
| ✓ | BbhMovieRecommendation65BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT609 |
| ✓ | BbhMovieRecommendation66BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT2 264 |
| ✓ | BbhMovieRecommendation67BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 516 |
| ✓ | BbhMovieRecommendation68BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT843 |
| ✓ | BbhMovieRecommendation69BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT882 |
| ✓ | BbhMovieRecommendation70BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT668 |
| ✓ | BbhMovieRecommendation71BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 555 |
| ✓ | BbhMovieRecommendation72BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 595 |
| ✓ | BbhMovieRecommendation73BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 029 |
| ✓ | BbhMovieRecommendation74BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 006 |
| ✓ | BbhMovieRecommendation75BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT685 |
| ✓ | BbhMovieRecommendation76BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT962 |
| ✓ | BbhMovieRecommendation77BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT577 |
| ✓ | BbhMovieRecommendation78BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 781 |
| ✓ | BbhMovieRecommendation79BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT2 381 |
| ✓ | BbhMovieRecommendation80BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT1 380 |
| ✓ | BbhMovieRecommendation81BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT622 |
| ✓ | BbhMovieRecommendation82BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT877 |
| ✓ | BbhMovieRecommendation83BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT966 |
| ✓ | BbhMovieRecommendation84BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT685 |
| ✓ | BbhMovieRecommendation85BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT2 631 |
| ✓ | BbhMovieRecommendation86BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT656 |
| ✓ | BbhMovieRecommendation87BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 066 |
| ✓ | BbhMovieRecommendation88BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT1 465 |
| ✓ | BbhMovieRecommendation89BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT583 |
| ✓ | BbhMovieRecommendation90BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT759 |
| ✓ | BbhMovieRecommendation91BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT625 |
| ✓ | BbhMovieRecommendation92BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT2 194 |
| ✓ | BbhMovieRecommendation93BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 384 |
| ✓ | BbhMovieRecommendation94BTpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 992 |
Reasoning - BBH movie recommendation (set 4)
29 passed3 failed |
90.6% |
IN32 296 OUT42 470 | |
| ✓ | BbhMovieRecommendation100BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 251 |
| ✓ | BbhMovieRecommendation101BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT713 |
| ✓ | BbhMovieRecommendation102BUFind a movie similar to The Mask, Batman, Pretty Woman, The Fugitive:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT592 |
| ✓ | BbhMovieRecommendation103BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT709 |
| ✓ | BbhMovieRecommendation104BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT1 006 |
| ✓ | BbhMovieRecommendation105BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT2 171 |
| ✓ | BbhMovieRecommendation106BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 459 |
| ✓ | BbhMovieRecommendation107BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT678 |
| ✓ | BbhMovieRecommendation108BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT2 294 |
| ✓ | BbhMovieRecommendation109BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT824 |
| ✓ | BbhMovieRecommendation110BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT2 189 |
| ✓ | BbhMovieRecommendation111BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT4 107 |
| ✓ | BbhMovieRecommendation112BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT770 |
| ✓ | BbhMovieRecommendation113BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT1 271 |
| ✓ | BbhMovieRecommendation114BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT684 |
| ✓ | BbhMovieRecommendation115BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT2 054 |
| × | BbhMovieRecommendation116BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation116BU: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 020 OUT765 |
| ✓ | BbhMovieRecommendation117BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT658 |
| × | BbhMovieRecommendation118BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation118BU: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 000 OUT3 526 |
| ✓ | BbhMovieRecommendation119BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT848 |
| ✓ | BbhMovieRecommendation120BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 649 |
| ✓ | BbhMovieRecommendation121BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT828 |
| ✓ | BbhMovieRecommendation122BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 097 |
| ✓ | BbhMovieRecommendation123BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT658 |
| ✓ | BbhMovieRecommendation124BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT988 |
| ✓ | BbhMovieRecommendation125BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 087 |
| ✓ | BbhMovieRecommendation126BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT637 |
| ✓ | BbhMovieRecommendation95BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT657 |
| ✓ | BbhMovieRecommendation96BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT764 |
| ✓ | BbhMovieRecommendation97BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT849 |
| ✓ | BbhMovieRecommendation98BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 220 |
| × | BbhMovieRecommendation99BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhMovieRecommendation99BU: expected (B), got [b babe]
Values:
got = "b babe" |
Failed | IN996 OUT3 467 |
Reasoning - BBH movie recommendation (set 5)
30 passed2 failed |
93.8% |
IN32 324 OUT38 446 | |
| ✓ | BbhMovieRecommendation127BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT617 |
| ✓ | BbhMovieRecommendation128BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 307 |
| ✓ | BbhMovieRecommendation129BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 325 |
| ✓ | BbhMovieRecommendation130BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 566 |
| ✓ | BbhMovieRecommendation131BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 412 |
| × | BbhMovieRecommendation132BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation132BU: expected (D), got [a]
Values:
got = "a" |
Failed | IN1 021 OUT1 545 |
| ✓ | BbhMovieRecommendation133BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 828 |
| ✓ | BbhMovieRecommendation134BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT752 |
| ✓ | BbhMovieRecommendation135BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT3 015 |
| ✓ | BbhMovieRecommendation136BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT2 401 |
| ✓ | BbhMovieRecommendation137BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT590 |
| ✓ | BbhMovieRecommendation138BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 717 |
| ✓ | BbhMovieRecommendation139BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT2 320 |
| ✓ | BbhMovieRecommendation140BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT923 |
| ✓ | BbhMovieRecommendation141BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 379 |
| ✓ | BbhMovieRecommendation142BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT727 |
| ✓ | BbhMovieRecommendation143BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT842 |
| ✓ | BbhMovieRecommendation144BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT2 033 |
| ✓ | BbhMovieRecommendation145BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 021 |
| ✓ | BbhMovieRecommendation146BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 085 |
| ✓ | BbhMovieRecommendation147BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT774 |
| ✓ | BbhMovieRecommendation148BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT673 |
| ✓ | BbhMovieRecommendation149BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT475 |
| ✓ | BbhMovieRecommendation150BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT705 |
| ✓ | BbhMovieRecommendation151BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT731 |
| ✓ | BbhMovieRecommendation152BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT514 |
| ✓ | BbhMovieRecommendation153BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT575 |
| ✓ | BbhMovieRecommendation154BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 010 |
| × | BbhMovieRecommendation155BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation155BU: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 009 OUT682 |
| ✓ | BbhMovieRecommendation156BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT924 |
| ✓ | BbhMovieRecommendation157BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 766 |
| ✓ | BbhMovieRecommendation158BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT1 212 |
Reasoning - BBH movie recommendation (set 6)
29 passed3 failed |
90.6% |
IN32 264 OUT45 884 | |
| ✓ | BbhMovieRecommendation159BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT894 |
| ✓ | BbhMovieRecommendation160BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 411 |
| ✓ | BbhMovieRecommendation161BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT785 |
| ✓ | BbhMovieRecommendation162BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 428 |
| × | BbhMovieRecommendation163BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("Monsters, Inc")
Context: BbhMovieRecommendation163BU: expected Monsters, Inc, got [a]
Values:
got = "a" |
Failed | IN1 007 OUT3 644 |
| ✓ | BbhMovieRecommendation164BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT735 |
| ✓ | BbhMovieRecommendation165BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT820 |
| ✓ | BbhMovieRecommendation166BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 609 |
| ✓ | BbhMovieRecommendation167BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 566 |
| ✓ | BbhMovieRecommendation168BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT722 |
| ✓ | BbhMovieRecommendation169BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT2 027 |
| ✓ | BbhMovieRecommendation170BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT921 |
| ✓ | BbhMovieRecommendation171BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT708 |
| ✓ | BbhMovieRecommendation172BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT749 |
| ✓ | BbhMovieRecommendation173BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 359 |
| ✓ | BbhMovieRecommendation174BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT2 072 |
| ✓ | BbhMovieRecommendation175BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT741 |
| ✓ | BbhMovieRecommendation176BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT795 |
| × | BbhMovieRecommendation177BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhMovieRecommendation177BU: expected (D), got [a]
Values:
got = "a" |
Failed | IN1 009 OUT4 130 |
| ✓ | BbhMovieRecommendation178BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 263 |
| ✓ | BbhMovieRecommendation179BUFind a movie similar to American Beauty, Pi, X-Men, The Godfather:1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT704 |
| ✓ | BbhMovieRecommendation180BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT1 158 |
| ✓ | BbhMovieRecommendation181BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 187 |
| ✓ | BbhMovieRecommendation182BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 877 |
| ✓ | BbhMovieRecommendation183BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 640 |
| ✓ | BbhMovieRecommendation184BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT789 |
| ✓ | BbhMovieRecommendation185BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT2 634 |
| ✓ | BbhMovieRecommendation186BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT3 775 |
| ✓ | BbhMovieRecommendation187BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT697 |
| × | BbhMovieRecommendation188BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhMovieRecommendation188BU: expected (B), got [d]
Values:
got = "d" |
Failed | IN1 008 OUT1 297 |
| ✓ | BbhMovieRecommendation189BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 031 |
| ✓ | BbhMovieRecommendation190BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT716 |
Reasoning - BBH movie recommendation (set 7)
29 passed3 failed |
90.6% |
IN32 336 OUT34 851 | |
| ✓ | BbhMovieRecommendation191BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT821 |
| ✓ | BbhMovieRecommendation192BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT2 026 |
| ✓ | BbhMovieRecommendation193BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT818 |
| ✓ | BbhMovieRecommendation194BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT648 |
| × | BbhMovieRecommendation195BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhMovieRecommendation195BU: expected (A), got [d]
Values:
got = "d" |
Failed | IN998 OUT1 184 |
| ✓ | BbhMovieRecommendation196BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT648 |
| ✓ | BbhMovieRecommendation197BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT711 |
| ✓ | BbhMovieRecommendation198BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT733 |
| ✓ | BbhMovieRecommendation199BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT732 |
| ✓ | BbhMovieRecommendation200BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT1 806 |
| ✓ | BbhMovieRecommendation201BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT685 |
| ✓ | BbhMovieRecommendation202BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT687 |
| ✓ | BbhMovieRecommendation203BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT582 |
| ✓ | BbhMovieRecommendation204BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT836 |
| ✓ | BbhMovieRecommendation205BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 015 |
| ✓ | BbhMovieRecommendation206BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 724 |
| ✓ | BbhMovieRecommendation207BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 169 |
| ✓ | BbhMovieRecommendation208BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT1 154 |
| ✓ | BbhMovieRecommendation209BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT638 |
| ✓ | BbhMovieRecommendation210BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT3 132 |
| ✓ | BbhMovieRecommendation211BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT1 568 |
| ✓ | BbhMovieRecommendation212BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 178 |
| ✓ | BbhMovieRecommendation213BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT548 |
| ✓ | BbhMovieRecommendation214BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT682 |
| ✓ | BbhMovieRecommendation215BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT857 |
| ✓ | BbhMovieRecommendation216BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT601 |
| ✓ | BbhMovieRecommendation217BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 615 |
| ✓ | BbhMovieRecommendation218BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT841 |
| × | BbhMovieRecommendation219BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhMovieRecommendation219BU: expected (A), got [d]
Values:
got = "d" |
Failed | IN1 003 OUT1 868 |
| ✓ | BbhMovieRecommendation220BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 021 |
| ✓ | BbhMovieRecommendation221BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT687 |
| × | BbhMovieRecommendation222BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation222BU: expected (C), got [b]
Values:
got = "b" |
Failed | IN1 006 OUT1 636 |
Reasoning - BBH movie recommendation (set 8)
26 passed4 failed |
86.7% |
IN30 201 OUT29 346 | |
| × | BbhMovieRecommendation223BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhMovieRecommendation223BU: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 007 OUT1 656 |
| ✓ | BbhMovieRecommendation224BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 315 |
| ✓ | BbhMovieRecommendation225BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT616 |
| ✓ | BbhMovieRecommendation226BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT1 353 |
| ✓ | BbhMovieRecommendation227BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 400 |
| ✓ | BbhMovieRecommendation228BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT723 |
| ✓ | BbhMovieRecommendation229BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT661 |
| ✓ | BbhMovieRecommendation230BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 395 |
| ✓ | BbhMovieRecommendation231BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT779 |
| ✓ | BbhMovieRecommendation232BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT752 |
| × | BbhMovieRecommendation233BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation233BU: expected (C), got [b]
Values:
got = "b" |
Failed | IN1 011 OUT730 |
| ✓ | BbhMovieRecommendation234BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT574 |
| ✓ | BbhMovieRecommendation235BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT684 |
| ✓ | BbhMovieRecommendation236BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 063 |
| ✓ | BbhMovieRecommendation237BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT768 |
| ✓ | BbhMovieRecommendation238BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT747 |
| ✓ | BbhMovieRecommendation239BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 057 |
| ✓ | BbhMovieRecommendation240BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT737 |
| ✓ | BbhMovieRecommendation241BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT658 |
| ✓ | BbhMovieRecommendation242BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT2 239 |
| ✓ | BbhMovieRecommendation243BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT834 |
| ✓ | BbhMovieRecommendation244BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT799 |
| ✓ | BbhMovieRecommendation245BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT741 |
| × | BbhMovieRecommendation246BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhMovieRecommendation246BU: expected (A), got [c]
Values:
got = "c" |
Failed | IN1 014 OUT2 758 |
| × | BbhMovieRecommendation247BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhMovieRecommendation247BU: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 007 OUT1 347 |
| ✓ | BbhMovieRecommendation248BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT842 |
| ✓ | BbhMovieRecommendation249BUpick the most similar movie1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT634 |
| ✓ | BbhMultistepArithmeticTwo10BU((6 - 0 * 5 + -3) * (6 - -7 + -2 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT392 |
| ✓ | BbhMultistepArithmeticTwo8BU((-7 * -9 + 8 * -3) * (5 + -7 - 4 * -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT619 |
| ✓ | BbhMultistepArithmeticTwo9BU((-9 - 1 * 5 * -5) - (6 + -3 - -1 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT473 |
Reasoning - BBH multi-step arithmetic (set 1)
32 passed |
100.0% |
IN31 392 OUT14 723 | |
| ✓ | BbhMultistepArithmeticTwo11BV((2 - -2 + -7 * 8) * (-7 * -8 * 3 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT943 |
| ✓ | BbhMultistepArithmeticTwo12BV((8 - 2 + -2 * 6) * (8 + -6 + -8 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT366 |
| ✓ | BbhMultistepArithmeticTwo13BV((-6 + -9 - -6 + -4) * (-1 - -6 + -4 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT469 |
| ✓ | BbhMultistepArithmeticTwo14BV((-5 - 4 * -8 + 8) * (4 + 3 - 9 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT572 |
| ✓ | BbhMultistepArithmeticTwo15BV((-5 * -7 * -6 + 9) * (-2 - 8 + -5 + 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT412 |
| ✓ | BbhMultistepArithmeticTwo16BV((8 + 9 - 4 - -9) + (8 + 7 - 6 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT352 |
| ✓ | BbhMultistepArithmeticTwo17BV((5 * -1 + -6 * -3) + (-1 + -8 - 5 + 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT363 |
| ✓ | BbhMultistepArithmeticTwo18BV((-6 - 6 + 7 - 7) - (5 + 3 - 9 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT374 |
| ✓ | BbhMultistepArithmeticTwo19BV((7 - 4 + -3 * 4) - (5 + -8 - 6 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT424 |
| ✓ | BbhMultistepArithmeticTwo20BV((-6 * -1 - 2 + -2) + (9 - 4 + -1 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT298 |
| ✓ | BbhMultistepArithmeticTwo21BV((8 * -6 + 6 * 1) - (-3 * 7 * 0 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT405 |
| ✓ | BbhMultistepArithmeticTwo22BV((-6 - 8 - -7 * -2) - (-9 - 5 + 7 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT361 |
| ✓ | BbhMultistepArithmeticTwo23BV((1 - 7 - -8 * 3) + (-7 - -2 + -3 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT456 |
| ✓ | BbhMultistepArithmeticTwo24BV((5 * -8 - -5 * -9) * (2 - -7 * 6 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT893 |
| ✓ | BbhMultistepArithmeticTwo25BV((6 + 1 - 4 - 3) - (-4 * -6 * -3 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT381 |
| ✓ | BbhMultistepArithmeticTwo26BV((-9 - -9 + 0 + -3) + (-2 - -1 - 1 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT344 |
| ✓ | BbhMultistepArithmeticTwo27BV((9 - 3 + 2 + -1) - (5 - -1 - -6 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT811 |
| ✓ | BbhMultistepArithmeticTwo28BV((-8 + 6 * -2 + 4) * (-4 * 5 + 2 - 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT589 |
| ✓ | BbhMultistepArithmeticTwo29BV((3 + -1 * 7 * -6) - (-7 * -1 + -5 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT375 |
| ✓ | BbhMultistepArithmeticTwo30BV((3 - 3 - -3 + -2) + (8 - 2 + -8 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT383 |
| ✓ | BbhMultistepArithmeticTwo31BV((-2 + -9 - -2 * 2) - (6 * 1 * -2 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT411 |
| ✓ | BbhMultistepArithmeticTwo32BV((-7 + -2 * 1 - 6) + (3 + 9 * 7 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT390 |
| ✓ | BbhMultistepArithmeticTwo33BV((4 + 7 * 4 - -5) - (-4 - 1 - -4 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT556 |
| ✓ | BbhMultistepArithmeticTwo34BV((1 - 6 * 0 - -7) * (6 * 0 + 6 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT385 |
| ✓ | BbhMultistepArithmeticTwo35BV((-1 + -7 + -2 - 7) * (6 - 1 - -1 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT356 |
| ✓ | BbhMultistepArithmeticTwo36BV((-5 + 6 - -5 + -6) + (-4 + -6 - 3 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT407 |
| ✓ | BbhMultistepArithmeticTwo37BV((0 - -1 + -1 + -3) - (-3 * 0 * 8 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT351 |
| ✓ | BbhMultistepArithmeticTwo38BV((-8 + -6 * 9 - 7) + (8 + 2 * -9 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT499 |
| ✓ | BbhMultistepArithmeticTwo39BV((-3 * 0 * 4 - -7) + (-9 * -6 - -8 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT373 |
| ✓ | BbhMultistepArithmeticTwo40BV((-5 - 2 + 9 * -1) * (7 - 5 + 9 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT503 |
| ✓ | BbhMultistepArithmeticTwo41BV((5 - 7 - 8 - 6) * (-2 * -8 * -9 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT535 |
| ✓ | BbhMultistepArithmeticTwo42BV((-5 + 1 - 2 - -3) * (-9 * 3 + 9 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT386 |
Reasoning - BBH multi-step arithmetic (set 2)
32 passed |
100.0% |
IN31 392 OUT15 279 | |
| ✓ | BbhMultistepArithmeticTwo43BV((-2 * -8 * 9 + -5) + (7 - -1 - -4 - 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT398 |
| ✓ | BbhMultistepArithmeticTwo44BV((-2 - 5 * -5 - 6) + (-5 * -4 + 2 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT727 |
| ✓ | BbhMultistepArithmeticTwo45BV((-5 - 6 * 5 + -9) + (1 + -4 - -2 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT1 029 |
| ✓ | BbhMultistepArithmeticTwo46BV((-8 + 0 * 0 * -1) - (7 + -1 + -3 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT402 |
| ✓ | BbhMultistepArithmeticTwo47BV((-9 - -3 * 1 - -3) * (-9 - 5 - -2 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT514 |
| ✓ | BbhMultistepArithmeticTwo48BV((5 * -2 + 8 * 1) * (9 + -3 * -8 - -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT518 |
| ✓ | BbhMultistepArithmeticTwo49BV((-7 * 9 - -9 + 7) + (9 * -6 * -4 * -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT529 |
| ✓ | BbhMultistepArithmeticTwo50BV((4 - 3 * -1 * 3) * (3 + 8 + 1 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT339 |
| ✓ | BbhMultistepArithmeticTwo51BV((-3 - 8 * 5 + 5) - (-1 * -3 - -2 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT461 |
| ✓ | BbhMultistepArithmeticTwo52BV((1 + 3 - -5 - 4) + (-3 - -3 + 6 * 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT399 |
| ✓ | BbhMultistepArithmeticTwo53BV((3 * 9 - 3 * 2) + (-1 * -7 - 1 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT377 |
| ✓ | BbhMultistepArithmeticTwo54BV((7 * 9 * 3 * 6) * (-8 * -6 + -9 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT700 |
| ✓ | BbhMultistepArithmeticTwo55BV((1 + 1 + -7 * -6) - (4 * 2 * -5 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT395 |
| ✓ | BbhMultistepArithmeticTwo56BV((1 + 1 * 3 - 2) * (-8 - -2 - 9 + -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT424 |
| ✓ | BbhMultistepArithmeticTwo57BV((5 + 6 * 4 * 9) + (-9 - 0 * 3 - 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT548 |
| ✓ | BbhMultistepArithmeticTwo58BV((-1 - 9 + -9 * -9) - (1 + -3 - 5 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT368 |
| ✓ | BbhMultistepArithmeticTwo59BV((-1 + -6 - -4 + -6) - (-2 * -5 - -1 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT352 |
| ✓ | BbhMultistepArithmeticTwo60BV((-2 - -7 + 5 * -3) + (2 - 8 + 5 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT349 |
| ✓ | BbhMultistepArithmeticTwo61BV((3 - -6 - 7 + 3) + (-7 + 6 * 9 + 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT424 |
| ✓ | BbhMultistepArithmeticTwo62BV((-1 - -9 * 8 - -9) + (-9 - 2 * -6 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT402 |
| ✓ | BbhMultistepArithmeticTwo63BV((-5 + 6 + -6 * -1) + (-5 + -5 + 9 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT361 |
| ✓ | BbhMultistepArithmeticTwo64BV((4 + -1 + -7 + 2) - (-7 + -9 * -6 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT405 |
| ✓ | BbhMultistepArithmeticTwo65BV((8 * 9 * 0 - -1) - (-9 - -7 + -4 - 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT417 |
| ✓ | BbhMultistepArithmeticTwo66BV((4 + 5 * -9 * -8) - (-9 + -7 - -6 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT524 |
| ✓ | BbhMultistepArithmeticTwo67BV((-8 + 5 - 0 - 6) * (-5 * -4 - 6 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT401 |
| ✓ | BbhMultistepArithmeticTwo68BV((-4 + -5 * 7 + 7) + (3 * -3 - -4 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT508 |
| ✓ | BbhMultistepArithmeticTwo69BV((0 * 8 - -9 * 6) - (-3 * -9 * -1 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT1 012 |
| ✓ | BbhMultistepArithmeticTwo70BV((0 * 9 + -8 * 7) - (7 - 9 + 1 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT391 |
| ✓ | BbhMultistepArithmeticTwo71BV((9 * 4 - 6 - -8) + (6 - 8 - -4 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT485 |
| ✓ | BbhMultistepArithmeticTwo72BV((6 - 1 - -3 + -2) + (5 * 5 + -1 - -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT304 |
| ✓ | BbhMultistepArithmeticTwo73BV((-4 + 4 + 3 * 1) - (-9 - -3 + -6 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT422 |
| ✓ | BbhMultistepArithmeticTwo74BV((-3 - -6 - -7 + 3) + (5 * 5 * 1 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT394 |
Reasoning - BBH multi-step arithmetic (set 3)
32 passed |
100.0% |
IN31 392 OUT17 038 | |
| ✓ | BbhMultistepArithmeticTwo100BV((7 - 5 * 7 + -1) + (6 * 4 - 9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT471 |
| ✓ | BbhMultistepArithmeticTwo101BV((6 + -8 * -3 - -5) * (2 - 7 * -2 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT1 024 |
| ✓ | BbhMultistepArithmeticTwo102BV((-1 * -1 * -6 - 7) - (-4 * 0 - -4 - 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT372 |
| ✓ | BbhMultistepArithmeticTwo103BV((-6 + 1 - -4 * -2) + (4 - -8 - -1 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT511 |
| ✓ | BbhMultistepArithmeticTwo104BV((7 + -9 * -5 + -3) * (-8 * 8 - -5 + -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT547 |
| ✓ | BbhMultistepArithmeticTwo105BV((3 * 4 * 0 * -3) - (6 + -4 + 2 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT346 |
| ✓ | BbhMultistepArithmeticTwo106BV((2 + -6 * -7 - 7) + (-2 - -5 + -4 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT408 |
| ✓ | BbhMultistepArithmeticTwo75BV((-8 * 4 + 7 - 5) + (9 + -6 + -2 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT586 |
| ✓ | BbhMultistepArithmeticTwo76BV((-8 - -6 * 7 * 0) * (0 + -5 * -8 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT623 |
| ✓ | BbhMultistepArithmeticTwo77BV((5 * 1 * 3 - 9) * (-7 + 4 - -7 * -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT1 807 |
| ✓ | BbhMultistepArithmeticTwo78BV((5 - -2 * -9 + 8) - (3 - 4 * -3 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT1 037 |
| ✓ | BbhMultistepArithmeticTwo79BV((7 + 1 + 9 * 1) + (7 - -3 + 3 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT434 |
| ✓ | BbhMultistepArithmeticTwo80BV((-6 + -2 * 3 * 0) + (3 + -4 - 7 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT386 |
| ✓ | BbhMultistepArithmeticTwo81BV((-8 + -4 * -9 * 0) + (2 * -9 + 3 * -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT368 |
| ✓ | BbhMultistepArithmeticTwo82BV((-8 + 6 - 8 * -9) - (9 - 9 * -2 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT424 |
| ✓ | BbhMultistepArithmeticTwo83BV((9 - -6 * 9 * 4) - (6 - 4 * -8 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT463 |
| ✓ | BbhMultistepArithmeticTwo84BV((8 + 3 * -8 - -8) - (2 + 2 * -9 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT473 |
| ✓ | BbhMultistepArithmeticTwo85BV((-9 + 8 - -4 * 2) - (1 - 9 + -6 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT512 |
| ✓ | BbhMultistepArithmeticTwo86BV((-1 * -6 - -1 - -8) - (8 + -7 * 8 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT438 |
| ✓ | BbhMultistepArithmeticTwo87BV((-3 * 6 - 0 * 6) - (-6 * -6 * 7 + -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT476 |
| ✓ | BbhMultistepArithmeticTwo88BV((-6 - -1 - -5 * -5) + (8 + -5 - 9 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT531 |
| ✓ | BbhMultistepArithmeticTwo89BV((-9 + -4 + 0 * -1) * (4 * -3 * -8 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT375 |
| ✓ | BbhMultistepArithmeticTwo90BV((-2 - 2 * 2 + 0) - (7 - 6 - -7 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT397 |
| ✓ | BbhMultistepArithmeticTwo91BV((-7 - -4 * 2 - 9) * (9 - 6 - 1 + -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT557 |
| ✓ | BbhMultistepArithmeticTwo92BV((-9 * 4 * -9 - -9) * (5 * 2 * 6 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT472 |
| ✓ | BbhMultistepArithmeticTwo93BV((-9 + -3 + 6 - 8) - (-7 * -2 * -6 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT421 |
| ✓ | BbhMultistepArithmeticTwo94BV((-7 + 5 * -6 * -6) * (7 * -5 + -6 - 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT607 |
| ✓ | BbhMultistepArithmeticTwo95BV((-8 * 0 + 2 * 2) * (-9 - 1 * -9 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT435 |
| ✓ | BbhMultistepArithmeticTwo96BV((9 + 2 + 3 - -3) + (-5 + 6 - -6 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT365 |
| ✓ | BbhMultistepArithmeticTwo97BV((5 - 6 - 7 + 6) - (-1 - 1 * -7 + 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT426 |
| ✓ | BbhMultistepArithmeticTwo98BV((-9 - -7 + 2 + 2) + (-7 + -9 - -1 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT404 |
| ✓ | BbhMultistepArithmeticTwo99BV((-6 * -4 + -1 + 1) + (-9 * -6 + 7 + 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT342 |
Reasoning - BBH multi-step arithmetic (set 4)
31 passed |
100.0% |
IN30 411 OUT15 619 | |
| ✓ | BbhMultistepArithmeticTwo107BV((-3 + -4 * 5 * -7) - (-1 * -4 + -9 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT449 |
| ✓ | BbhMultistepArithmeticTwo108BV((0 + -9 * 6 * 4) + (5 - -3 + 8 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT399 |
| ✓ | BbhMultistepArithmeticTwo109BV((-4 - -6 - -5 - 5) - (-2 + -4 - -8 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT423 |
| ✓ | BbhMultistepArithmeticTwo110BV((-3 + -4 * 3 - -8) + (8 - 1 - -4 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT452 |
| ✓ | BbhMultistepArithmeticTwo111BV((-4 - 5 + 2 + 7) - (-9 * 6 * -1 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT325 |
| ✓ | BbhMultistepArithmeticTwo112BV((8 + 1 + 6 + 6) - (7 - -7 - 9 - -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT322 |
| ✓ | BbhMultistepArithmeticTwo113BV((-9 * -5 - 5 + 7) - (-3 - 8 + 9 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT546 |
| ✓ | BbhMultistepArithmeticTwo114BV((0 + 1 - -6 * 9) * (4 + 7 + -9 - 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT485 |
| ✓ | BbhMultistepArithmeticTwo115BV((-1 + -6 + -3 * -2) * (-9 * -1 + 2 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT391 |
| ✓ | BbhMultistepArithmeticTwo116BV((6 * 5 + 0 - -3) * (4 * 5 * 3 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT413 |
| ✓ | BbhMultistepArithmeticTwo117BV((-2 * -1 + 1 * -3) - (7 * -7 + -5 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT332 |
| ✓ | BbhMultistepArithmeticTwo118BV((-1 + 5 - -1 + 3) + (0 - -3 - -5 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT416 |
| ✓ | BbhMultistepArithmeticTwo119BV((0 - 5 + 0 + 3) * (-7 + -8 * 4 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT441 |
| ✓ | BbhMultistepArithmeticTwo120BV((-2 - 3 - 0 - 2) * (1 - -7 - -3 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT422 |
| ✓ | BbhMultistepArithmeticTwo121BV((3 + 3 - -4 - 2) + (-2 - -7 * -2 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT795 |
| ✓ | BbhMultistepArithmeticTwo122BV((-9 - -8 + 3 * -1) * (-9 * -2 * 4 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT455 |
| ✓ | BbhMultistepArithmeticTwo123BV((-1 - 9 * 6 + 3) * (0 - 8 - -4 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT371 |
| ✓ | BbhMultistepArithmeticTwo124BV((-9 - 8 * 9 - 0) + (-4 + 4 + -2 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT364 |
| ✓ | BbhMultistepArithmeticTwo125BV((-6 - -5 - -9 - -9) * (-6 * 4 - 8 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT568 |
| ✓ | BbhMultistepArithmeticTwo126BV((1 + 2 * -1 - -8) - (-8 - 2 - -1 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT434 |
| ✓ | BbhMultistepArithmeticTwo127BV((-6 * -2 + -2 - -9) + (-9 * 3 * -1 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT364 |
| ✓ | BbhMultistepArithmeticTwo128BV((-1 * -3 - 2 + -7) + (4 * 8 * -6 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT430 |
| ✓ | BbhMultistepArithmeticTwo129BV((8 + 6 * -1 - -3) * (-5 * 0 - -7 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT570 |
| ✓ | BbhMultistepArithmeticTwo130BV((-5 - -1 - 7 * 8) * (-1 - -4 * -7 * 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT649 |
| ✓ | BbhMultistepArithmeticTwo131BV((4 - 8 - -9 * 7) - (5 + -8 * 9 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT535 |
| ✓ | BbhMultistepArithmeticTwo132BV((-7 - -8 + 5 - -4) + (6 - -6 * 4 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT1 357 |
| ✓ | BbhMultistepArithmeticTwo133BV((-3 * 4 * 3 + -5) + (-7 - 5 + 7 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT452 |
| ✓ | BbhMultistepArithmeticTwo134BV((-2 - 1 - -6 + -9) - (4 * 8 + -8 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT401 |
| ✓ | BbhMultistepArithmeticTwo135BV((-2 * 4 - 3 - 3) - (-9 * -3 - -8 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT1 361 |
| ✓ | BbhMultistepArithmeticTwo136BV((-4 + -6 + -6 - -9) + (-2 * -5 * 5 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT324 |
| ✓ | BbhMultistepArithmeticTwo137BV((-5 * 5 + 1 - 4) + (9 * 0 - -7 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT373 |
Reasoning - BBH multi-step arithmetic (set 5)
31 passed |
100.0% |
IN30 411 OUT14 205 | |
| ✓ | BbhMultistepArithmeticTwo138BV((1 - -8 + 0 - -3) + (-3 + -4 + 1 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT384 |
| ✓ | BbhMultistepArithmeticTwo139BV((8 - -5 + -9 + 7) + (-9 * -1 * -6 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT462 |
| ✓ | BbhMultistepArithmeticTwo140BV((0 * 3 - -9 + -1) - (-5 * -6 - 6 + 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT397 |
| ✓ | BbhMultistepArithmeticTwo141BV((-4 + -9 + -7 * 8) - (-8 + 0 * -5 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT394 |
| ✓ | BbhMultistepArithmeticTwo142BV((7 + -7 * 9 * 5) - (-2 + -6 - -8 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT569 |
| ✓ | BbhMultistepArithmeticTwo143BV((-4 - -8 + 2 + -8) * (-3 * 0 + 4 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT335 |
| ✓ | BbhMultistepArithmeticTwo144BV((3 - 9 - 3 - 4) + (1 * -2 * -3 - -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT296 |
| ✓ | BbhMultistepArithmeticTwo145BV((-6 * -4 + 1 + -1) * (-1 + 2 * 7 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT438 |
| ✓ | BbhMultistepArithmeticTwo146BV((-9 * -3 - 8 - 5) + (1 * -2 - -4 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT1 301 |
| ✓ | BbhMultistepArithmeticTwo147BV((3 + -6 * -2 + 7) * (-6 - -4 - -6 + 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT490 |
| ✓ | BbhMultistepArithmeticTwo148BV((-7 * 2 + -4 * -8) - (-5 * 0 * 1 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT319 |
| ✓ | BbhMultistepArithmeticTwo149BV((-4 * 4 + -1 * 5) - (-6 + 1 - -8 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT390 |
| ✓ | BbhMultistepArithmeticTwo150BV((7 - -9 + 0 - 0) + (-9 + 5 - -9 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT423 |
| ✓ | BbhMultistepArithmeticTwo151BV((4 + 3 + -9 + -2) * (7 * 2 + 8 - 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT413 |
| ✓ | BbhMultistepArithmeticTwo152BV((3 - 1 + -3 * -4) - (8 * 9 - -7 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT478 |
| ✓ | BbhMultistepArithmeticTwo153BV((-6 * -7 * -8 - -2) + (-5 - 2 + 0 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT369 |
| ✓ | BbhMultistepArithmeticTwo154BV((5 * 7 * -4 + 7) * (-5 - -6 + -8 * -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT562 |
| ✓ | BbhMultistepArithmeticTwo155BV((4 - 2 * 2 + 3) + (-3 - -9 - -4 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT423 |
| ✓ | BbhMultistepArithmeticTwo156BV((8 - 7 - -3 + -9) - (-6 * -6 + -4 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT517 |
| ✓ | BbhMultistepArithmeticTwo157BV((-7 + 9 + -9 * -4) - (-7 * 6 - 7 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT381 |
| ✓ | BbhMultistepArithmeticTwo158BV((-6 * -2 + 6 - -8) - (-2 * -4 * 8 + 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT432 |
| ✓ | BbhMultistepArithmeticTwo159BV((-1 + 5 * 4 + 9) - (-7 - 6 * -8 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT637 |
| ✓ | BbhMultistepArithmeticTwo160BV((-4 + -2 + -6 + -1) + (-4 * 2 + -5 * 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT304 |
| ✓ | BbhMultistepArithmeticTwo161BV((-8 - -1 - -2 * -7) + (3 + 2 - 3 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT440 |
| ✓ | BbhMultistepArithmeticTwo162BV((4 + 5 - 7 + 9) * (-1 - -4 + -7 - -5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT363 |
| ✓ | BbhMultistepArithmeticTwo163BV((7 + 7 * 3 - 3) - (-3 + 4 - -6 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT486 |
| ✓ | BbhMultistepArithmeticTwo164BV((-4 + 5 - 4 - -6) * (-3 * -1 * 2 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT331 |
| ✓ | BbhMultistepArithmeticTwo165BV((2 + 7 - -5 - 1) - (0 - 5 - 9 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT355 |
| ✓ | BbhMultistepArithmeticTwo166BV((4 * -4 + -6 - -2) - (9 + -7 + -8 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT575 |
| ✓ | BbhMultistepArithmeticTwo167BV((3 - 2 - 3 * 3) + (-8 * -1 * -9 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT470 |
| ✓ | BbhMultistepArithmeticTwo168BV((-1 * -5 - -3 + -9) + (-5 - 3 * -2 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT471 |
Reasoning - BBH multi-step arithmetic (set 6)
32 passed |
100.0% |
IN31 392 OUT15 356 | |
| ✓ | BbhMultistepArithmeticTwo169BW((5 * -1 - 7 * 5) - (6 - -5 * 3 * 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT712 |
| ✓ | BbhMultistepArithmeticTwo170BW((9 - -5 - 1 + -2) * (-5 * 3 - -5 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT439 |
| ✓ | BbhMultistepArithmeticTwo171BW((-8 - -3 * 5 + 4) * (2 * 7 - -3 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT527 |
| ✓ | BbhMultistepArithmeticTwo172BW((5 - 7 + -6 + 4) * (2 - -9 + 3 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT380 |
| ✓ | BbhMultistepArithmeticTwo173BW((-5 - -1 - 6 * 0) + (-3 + 5 * 0 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT353 |
| ✓ | BbhMultistepArithmeticTwo174BW((-3 - 9 - 3 - 1) + (-4 * -9 * -7 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT383 |
| ✓ | BbhMultistepArithmeticTwo175BW((1 * -1 + 5 * -4) + (-2 - 5 * 4 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT386 |
| ✓ | BbhMultistepArithmeticTwo176BW((-3 - -8 - -7 + -2) - (-2 - 8 + -1 - -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT493 |
| ✓ | BbhMultistepArithmeticTwo177BW((-5 * -6 + 3 * -3) - (-1 + -8 * 8 * -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT427 |
| ✓ | BbhMultistepArithmeticTwo178BW((-5 - -7 - 4 - 8) - (-4 + 4 + 1 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT338 |
| ✓ | BbhMultistepArithmeticTwo179BW((9 + 7 - 4 - 7) - (-5 + 6 + -1 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT307 |
| ✓ | BbhMultistepArithmeticTwo180BW((8 * -6 + -9 + 9) * (0 - 4 - 0 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT415 |
| ✓ | BbhMultistepArithmeticTwo181BW((-4 * 1 + 8 + 9) * (7 * -6 * -5 - 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT521 |
| ✓ | BbhMultistepArithmeticTwo182BW((-5 - 6 * 2 + 4) - (1 - 9 - 0 * 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT413 |
| ✓ | BbhMultistepArithmeticTwo183BW((-7 - 1 + 3 * -2) + (0 - 4 + 6 + 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT335 |
| ✓ | BbhMultistepArithmeticTwo184BW((7 * 2 + 4 - -6) * (6 - 7 - 7 * -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT624 |
| ✓ | BbhMultistepArithmeticTwo185BW((-6 * -9 + 1 * 7) - (0 + 3 * 5 - 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT377 |
| ✓ | BbhMultistepArithmeticTwo186BW((-8 - -1 + -8 + -3) * (-4 * 6 - 2 - 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT480 |
| ✓ | BbhMultistepArithmeticTwo187BW((0 * -4 - -7 * 5) + (-4 * 0 + 6 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT501 |
| ✓ | BbhMultistepArithmeticTwo188BW((9 * -9 + -4 * -8) - (5 * 0 + 8 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT234 |
| ✓ | BbhMultistepArithmeticTwo189BW((-6 + -9 - -6 + -1) + (-8 * 5 * -6 + 7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT492 |
| ✓ | BbhMultistepArithmeticTwo190BW((5 + 4 + 7 - 6) - (-8 * 0 - 9 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT383 |
| ✓ | BbhMultistepArithmeticTwo191BW((9 - 6 + -4 + -5) * (4 - -8 + -9 + 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT272 |
| ✓ | BbhMultistepArithmeticTwo192BW((-9 * 6 - -3 * -3) + (-4 - -9 + 4 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT434 |
| ✓ | BbhMultistepArithmeticTwo193BW((-8 * -8 - -5 * -6) * (3 + -9 - 5 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT992 |
| ✓ | BbhMultistepArithmeticTwo194BW((0 + 3 * 4 * 9) - (4 - -5 + -4 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT361 |
| ✓ | BbhMultistepArithmeticTwo195BW((-4 + -7 - 3 + 3) - (-3 + -2 - -3 * -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT864 |
| ✓ | BbhMultistepArithmeticTwo196BW((-7 * -2 * -1 + -9) - (-7 + 5 - 0 * -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT439 |
| ✓ | BbhMultistepArithmeticTwo197BW((-3 + 2 * -7 * 5) * (-9 - 9 - -9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT623 |
| ✓ | BbhMultistepArithmeticTwo198BW((1 + 0 + 2 - 4) + (-9 + 6 * -5 + 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT323 |
| ✓ | BbhMultistepArithmeticTwo199BW((-3 + -4 * 4 + -8) + (1 - -3 + 4 - -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT523 |
| ✓ | BbhMultistepArithmeticTwo200BW((3 * -7 * -4 * 2) + (7 - -3 - -3 * 1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT1 005 |
Reasoning - BBH multi-step arithmetic (set 7)
32 passed |
100.0% |
IN31 392 OUT13 916 | |
| ✓ | BbhMultistepArithmeticTwo201BW((-5 * -6 + -9 - 3) + (-7 - -2 - 6 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT503 |
| ✓ | BbhMultistepArithmeticTwo202BW((6 - 5 + 5 + -9) - (9 + 8 * -6 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT385 |
| ✓ | BbhMultistepArithmeticTwo203BW((8 * 3 - -1 - -3) * (-4 * 7 * -3 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT562 |
| ✓ | BbhMultistepArithmeticTwo204BW((5 + 3 - 6 + 1) + (-5 + 5 - 6 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT264 |
| ✓ | BbhMultistepArithmeticTwo205BW((-9 + 6 + 1 - 7) * (2 - -2 + -9 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT428 |
| ✓ | BbhMultistepArithmeticTwo206BW((1 * 0 - -5 - 3) + (9 * 6 + -4 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT315 |
| ✓ | BbhMultistepArithmeticTwo207BW((1 - 2 + 5 * 2) + (-2 - -7 + 3 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT499 |
| ✓ | BbhMultistepArithmeticTwo208BW((5 + -7 + 1 - 5) - (-3 * -1 * 7 + -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT371 |
| ✓ | BbhMultistepArithmeticTwo209BW((-7 * -2 + -4 + -4) - (9 * 1 * -6 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT368 |
| ✓ | BbhMultistepArithmeticTwo210BW((1 + 0 - 3 + 0) - (-7 - 9 * 1 + -3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT364 |
| ✓ | BbhMultistepArithmeticTwo211BW((7 * 6 - -2 - 7) * (4 * 5 * -3 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT441 |
| ✓ | BbhMultistepArithmeticTwo212BW((3 * -9 * 3 - 7) - (-9 + 4 * 8 + 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT415 |
| ✓ | BbhMultistepArithmeticTwo213BW((9 + 6 - 9 + 8) * (9 - -9 * -6 + -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT654 |
| ✓ | BbhMultistepArithmeticTwo214BW((5 * 6 + -6 - 8) * (-5 - 6 - 5 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT458 |
| ✓ | BbhMultistepArithmeticTwo215BW((5 + 4 - -1 * -1) + (-6 - -7 + -1 * 6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT504 |
| ✓ | BbhMultistepArithmeticTwo216BW((9 - 5 - 4 + -2) + (-9 + 1 - 1 - 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT305 |
| ✓ | BbhMultistepArithmeticTwo217BW((6 - 9 * -6 - -9) + (3 * 3 + -5 * -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT518 |
| ✓ | BbhMultistepArithmeticTwo218BW((-2 * 5 + -3 + 0) * (-9 + 2 + 5 * 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT357 |
| ✓ | BbhMultistepArithmeticTwo219BW((5 * -3 * 3 * 7) * (0 + -9 - -5 - -2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT361 |
| ✓ | BbhMultistepArithmeticTwo220BW((8 - 3 + 0 + 7) + (-9 * -6 * 2 - 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT276 |
| ✓ | BbhMultistepArithmeticTwo221BW((-2 * 9 * -8 + 8) - (0 * -4 - -6 * 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT834 |
| ✓ | BbhMultistepArithmeticTwo222BW((-7 * -6 * 0 - 5) * (-3 - 9 * -8 * -9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT674 |
| ✓ | BbhMultistepArithmeticTwo223BW((3 * 8 * -1 * -9) - (-2 + 2 * -8 * 8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT436 |
| ✓ | BbhMultistepArithmeticTwo224BW((-9 + 5 * 0 - -8) + (6 - -5 * -7 + -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT584 |
| ✓ | BbhMultistepArithmeticTwo225BW((-2 - -7 - 5 * 2) + (-3 - 5 * 3 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT420 |
| ✓ | BbhMultistepArithmeticTwo226BW((-9 + -7 + -1 - -1) * (3 - -7 - -1 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT375 |
| ✓ | BbhMultistepArithmeticTwo227BW((-1 + 6 + 9 + 3) - (-5 - 9 - -3 + 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT309 |
| ✓ | BbhMultistepArithmeticTwo228BW((-7 + 9 * 9 * 6) - (-2 - -5 + 9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT384 |
| ✓ | BbhMultistepArithmeticTwo229BW((-8 - -7 + 0 + 3) + (-3 + 5 - -5 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT390 |
| ✓ | BbhMultistepArithmeticTwo230BW((8 - 4 - -8 - -7) + (6 - 5 - -1 - 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT381 |
| ✓ | BbhMultistepArithmeticTwo231BW((-6 * 8 + -6 - -5) + (-4 - 9 * 5 - -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT426 |
| ✓ | BbhMultistepArithmeticTwo232BW((5 * -9 - 4 - 4) - (-3 * 2 * 9 + -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT355 |
Reasoning - BBH multi-step arithmetic (set 8)
31 passed1 failed |
96.9% |
IN31 751 OUT15 125 | |
| ✓ | BbhMultistepArithmeticTwo233BW((4 * -9 * 3 * -8) + (-9 - 4 - -9 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT454 |
| ✓ | BbhMultistepArithmeticTwo234BW((-7 - 2 - 0 + 2) * (-4 * -5 + -8 + 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT386 |
| ✓ | BbhMultistepArithmeticTwo235BW((-1 + 7 * 7 * -9) * (4 + 3 + -8 * 4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT646 |
| ✓ | BbhMultistepArithmeticTwo236BW((7 * 8 + 1 + 3) - (3 + -1 - -9 + -6)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT339 |
| ✓ | BbhMultistepArithmeticTwo237BW((3 + -9 + 8 + -4) * (-4 + -6 - 4 - -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT368 |
| ✓ | BbhMultistepArithmeticTwo238BW((1 * 5 * -8 - -2) * (-2 * 6 - -5 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT635 |
| ✓ | BbhMultistepArithmeticTwo239BW((-8 - -1 * -8 * -2) * (-6 + 8 + -5 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT497 |
| ✓ | BbhMultistepArithmeticTwo240BW((4 - -6 + -9 + -7) - (9 + -3 - -7 * -8)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT525 |
| ✓ | BbhMultistepArithmeticTwo241BW((0 * -7 - 6 * 3) * (-9 + -5 - 2 * -1)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT511 |
| ✓ | BbhMultistepArithmeticTwo242BW((-8 * 1 * -1 + -5) + (-2 - -3 + 1 * -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT395 |
| ✓ | BbhMultistepArithmeticTwo243BW((8 - -9 * 4 * 9) * (-4 * -3 * 7 * 9)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT984 |
| ✓ | BbhMultistepArithmeticTwo244BW((1 * -9 - 4 + -9) + (-1 + -6 + 9 - 5)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT355 |
| ✓ | BbhMultistepArithmeticTwo245BW((-1 + 2 - 9 + 0) * (0 - -2 + -6 - 2)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT387 |
| ✓ | BbhMultistepArithmeticTwo246BW((7 * -5 * -9 + 7) - (2 + -8 * 6 * 3)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT491 |
| ✓ | BbhMultistepArithmeticTwo247BW((-5 + -8 - -6 * 7) + (9 * -7 - -5 - -4)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT571 |
| ✓ | BbhMultistepArithmeticTwo248BW((-6 * 9 - -5 + 8) * (-8 * -6 - -1 * 0)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT621 |
| ✓ | BbhMultistepArithmeticTwo249BW((5 - -2 * 7 + -2) - (0 * 9 * -6 - -7)) =1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT489 |
| ✓ | BbhNavigate10BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT444 |
| ✓ | BbhNavigate11BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT523 |
| ✓ | BbhNavigate12BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT311 |
| × | BbhNavigate13BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("Yes")
Context: BbhNavigate13BW: expected Yes, got [no]
Values:
got = "no" |
Failed | IN1 019 OUT550 |
| ✓ | BbhNavigate14BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT193 |
| ✓ | BbhNavigate15BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT356 |
| ✓ | BbhNavigate16BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT200 |
| ✓ | BbhNavigate17BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT507 |
| ✓ | BbhNavigate18BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT1 015 |
| ✓ | BbhNavigate19BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT373 |
| ✓ | BbhNavigate20BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT346 |
| ✓ | BbhNavigate21BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT248 |
| ✓ | BbhNavigate22BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT317 |
| ✓ | BbhNavigate8BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT447 |
| ✓ | BbhNavigate9BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT641 |
Reasoning - BBH navigation - returns to start? (set 1)
32 passed |
100.0% |
IN32 212 OUT14 234 | |
| ✓ | BbhNavigate23BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT314 |
| ✓ | BbhNavigate24BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT208 |
| ✓ | BbhNavigate25BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT471 |
| ✓ | BbhNavigate26BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT723 |
| ✓ | BbhNavigate27BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 369 |
| ✓ | BbhNavigate28BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT271 |
| ✓ | BbhNavigate29BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT168 |
| ✓ | BbhNavigate30BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT433 |
| ✓ | BbhNavigate31BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT284 |
| ✓ | BbhNavigate32BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT399 |
| ✓ | BbhNavigate33BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT458 |
| ✓ | BbhNavigate34BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT432 |
| ✓ | BbhNavigate35BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT220 |
| ✓ | BbhNavigate36BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 534 |
| ✓ | BbhNavigate37BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT201 |
| ✓ | BbhNavigate38BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT396 |
| ✓ | BbhNavigate39BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT122 |
| ✓ | BbhNavigate40BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT203 |
| ✓ | BbhNavigate41BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT470 |
| ✓ | BbhNavigate42BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT448 |
| ✓ | BbhNavigate43BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT351 |
| ✓ | BbhNavigate44BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT209 |
| ✓ | BbhNavigate45BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT323 |
| ✓ | BbhNavigate46BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT315 |
| ✓ | BbhNavigate47BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT367 |
| ✓ | BbhNavigate48BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT153 |
| ✓ | BbhNavigate49BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 250 |
| ✓ | BbhNavigate50BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT618 |
| ✓ | BbhNavigate51BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT238 |
| ✓ | BbhNavigate52BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT667 |
| ✓ | BbhNavigate53BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT393 |
| ✓ | BbhNavigate54BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT226 |
Reasoning - BBH navigation - returns to start? (set 2)
30 passed |
100.0% |
IN30 187 OUT16 880 | |
| ✓ | BbhNavigate55BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT287 |
| ✓ | BbhNavigate56BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT459 |
| ✓ | BbhNavigate57BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT724 |
| ✓ | BbhNavigate58BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT313 |
| ✓ | BbhNavigate59BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT369 |
| ✓ | BbhNavigate60BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT1 181 |
| ✓ | BbhNavigate61BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT1 573 |
| ✓ | BbhNavigate62BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT1 575 |
| ✓ | BbhNavigate63BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 134 |
| ✓ | BbhNavigate64BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT200 |
| ✓ | BbhNavigate65BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT171 |
| ✓ | BbhNavigate66BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT423 |
| ✓ | BbhNavigate67BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT204 |
| ✓ | BbhNavigate68BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT413 |
| ✓ | BbhNavigate69BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT144 |
| ✓ | BbhNavigate70BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT214 |
| ✓ | BbhNavigate71BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT133 |
| ✓ | BbhNavigate72BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT349 |
| ✓ | BbhNavigate73BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT163 |
| ✓ | BbhNavigate74BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT755 |
| ✓ | BbhNavigate75BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT165 |
| ✓ | BbhNavigate76BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT559 |
| ✓ | BbhNavigate77BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT557 |
| ✓ | BbhNavigate78BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT316 |
| ✓ | BbhNavigate79BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT168 |
| ✓ | BbhNavigate80BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT576 |
| ✓ | BbhNavigate81BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 879 |
| ✓ | BbhNavigate82BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT385 |
| ✓ | BbhNavigate83BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT661 |
| ✓ | BbhNavigate84BWdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT830 |
Reasoning - BBH navigation - returns to start? (set 3)
32 passed |
100.0% |
IN32 213 OUT15 658 | |
| ✓ | BbhNavigate100CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT499 |
| ✓ | BbhNavigate101CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT300 |
| ✓ | BbhNavigate102CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT228 |
| ✓ | BbhNavigate103CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT399 |
| ✓ | BbhNavigate104CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT638 |
| ✓ | BbhNavigate105CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT395 |
| ✓ | BbhNavigate106CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT346 |
| ✓ | BbhNavigate107CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT170 |
| ✓ | BbhNavigate108CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT476 |
| ✓ | BbhNavigate109CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT746 |
| ✓ | BbhNavigate110CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT238 |
| ✓ | BbhNavigate111CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT1 309 |
| ✓ | BbhNavigate112CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT397 |
| ✓ | BbhNavigate113CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT296 |
| ✓ | BbhNavigate114CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT216 |
| ✓ | BbhNavigate115CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT310 |
| ✓ | BbhNavigate116CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT165 |
| ✓ | BbhNavigate85CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT804 |
| ✓ | BbhNavigate86CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 030 |
| ✓ | BbhNavigate87CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT425 |
| ✓ | BbhNavigate88CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT391 |
| ✓ | BbhNavigate89CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT227 |
| ✓ | BbhNavigate90CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT513 |
| ✓ | BbhNavigate91CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT346 |
| ✓ | BbhNavigate92CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 115 |
| ✓ | BbhNavigate93CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT618 |
| ✓ | BbhNavigate94CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT308 |
| ✓ | BbhNavigate95CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 187 |
| ✓ | BbhNavigate96CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT333 |
| ✓ | BbhNavigate97CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT568 |
| ✓ | BbhNavigate98CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT472 |
| ✓ | BbhNavigate99CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT193 |
Reasoning - BBH navigation - returns to start? (set 4)
32 passed |
100.0% |
IN32 201 OUT15 275 | |
| ✓ | BbhNavigate117CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT651 |
| ✓ | BbhNavigate118CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT190 |
| ✓ | BbhNavigate119CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT163 |
| ✓ | BbhNavigate120CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT492 |
| ✓ | BbhNavigate121CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT263 |
| ✓ | BbhNavigate122CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT487 |
| ✓ | BbhNavigate123CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT848 |
| ✓ | BbhNavigate124CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT665 |
| ✓ | BbhNavigate125CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT286 |
| ✓ | BbhNavigate126CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT1 654 |
| ✓ | BbhNavigate127CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT586 |
| ✓ | BbhNavigate128CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT261 |
| ✓ | BbhNavigate129CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT401 |
| ✓ | BbhNavigate130CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT390 |
| ✓ | BbhNavigate131CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT899 |
| ✓ | BbhNavigate132CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT639 |
| ✓ | BbhNavigate133CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT402 |
| ✓ | BbhNavigate134CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT127 |
| ✓ | BbhNavigate135CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT731 |
| ✓ | BbhNavigate136CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT694 |
| ✓ | BbhNavigate137CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT452 |
| ✓ | BbhNavigate138CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT878 |
| ✓ | BbhNavigate139CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT550 |
| ✓ | BbhNavigate140CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT221 |
| ✓ | BbhNavigate141CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT286 |
| ✓ | BbhNavigate142CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT342 |
| ✓ | BbhNavigate143CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT333 |
| ✓ | BbhNavigate144CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT259 |
| ✓ | BbhNavigate145CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT283 |
| ✓ | BbhNavigate146CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT341 |
| ✓ | BbhNavigate147CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT182 |
| ✓ | BbhNavigate148CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT319 |
Reasoning - BBH navigation - returns to start? (set 5)
32 passed |
100.0% |
IN32 138 OUT19 373 | |
| ✓ | BbhNavigate149CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT616 |
| ✓ | BbhNavigate150CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT664 |
| ✓ | BbhNavigate151CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT278 |
| ✓ | BbhNavigate152CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT133 |
| ✓ | BbhNavigate153CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 993 |
| ✓ | BbhNavigate154CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT393 |
| ✓ | BbhNavigate155CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT132 |
| ✓ | BbhNavigate156CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT483 |
| ✓ | BbhNavigate157CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT856 |
| ✓ | BbhNavigate158CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT674 |
| ✓ | BbhNavigate159CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT367 |
| ✓ | BbhNavigate160CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT226 |
| ✓ | BbhNavigate161CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT186 |
| ✓ | BbhNavigate162CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT1 735 |
| ✓ | BbhNavigate163CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT707 |
| ✓ | BbhNavigate164CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT333 |
| ✓ | BbhNavigate165CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT261 |
| ✓ | BbhNavigate166CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT1 800 |
| ✓ | BbhNavigate167CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT509 |
| ✓ | BbhNavigate168CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT491 |
| ✓ | BbhNavigate169CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT278 |
| ✓ | BbhNavigate170CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT230 |
| ✓ | BbhNavigate171CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT529 |
| ✓ | BbhNavigate172CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 324 |
| ✓ | BbhNavigate173CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT555 |
| ✓ | BbhNavigate174CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT272 |
| ✓ | BbhNavigate175CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT592 |
| ✓ | BbhNavigate176CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT890 |
| ✓ | BbhNavigate177CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT614 |
| ✓ | BbhNavigate178CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT358 |
| ✓ | BbhNavigate179CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT506 |
| ✓ | BbhNavigate180CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT388 |
Reasoning - BBH navigation - returns to start? (set 6)
32 passed |
100.0% |
IN32 216 OUT18 512 | |
| ✓ | BbhNavigate181CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT687 |
| ✓ | BbhNavigate182CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT967 |
| ✓ | BbhNavigate183CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT271 |
| ✓ | BbhNavigate184CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT834 |
| ✓ | BbhNavigate185CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT208 |
| ✓ | BbhNavigate186CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT378 |
| ✓ | BbhNavigate187CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT390 |
| ✓ | BbhNavigate188CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT430 |
| ✓ | BbhNavigate189CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT201 |
| ✓ | BbhNavigate190CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT179 |
| ✓ | BbhNavigate191CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT232 |
| ✓ | BbhNavigate192CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT768 |
| ✓ | BbhNavigate193CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT543 |
| ✓ | BbhNavigate194CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT159 |
| ✓ | BbhNavigate195CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT279 |
| ✓ | BbhNavigate196CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT2 340 |
| ✓ | BbhNavigate197CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT686 |
| ✓ | BbhNavigate198CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT504 |
| ✓ | BbhNavigate199CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 111 |
| ✓ | BbhNavigate200CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 757 |
| ✓ | BbhNavigate201CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT536 |
| ✓ | BbhNavigate202CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT352 |
| ✓ | BbhNavigate203CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT310 |
| ✓ | BbhNavigate204CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT484 |
| ✓ | BbhNavigate205CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT276 |
| ✓ | BbhNavigate206CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT325 |
| ✓ | BbhNavigate207CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT267 |
| ✓ | BbhNavigate208CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 164 |
| ✓ | BbhNavigate209CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT286 |
| ✓ | BbhNavigate210CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT154 |
| ✓ | BbhNavigate211CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT502 |
| ✓ | BbhNavigate212CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT932 |
Reasoning - BBH navigation - returns to start? (set 7)
30 passed |
100.0% |
IN30 113 OUT17 174 | |
| ✓ | BbhNavigate213CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT527 |
| ✓ | BbhNavigate214CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT1 192 |
| ✓ | BbhNavigate215CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT403 |
| ✓ | BbhNavigate216CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT268 |
| ✓ | BbhNavigate217CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT593 |
| ✓ | BbhNavigate218CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT451 |
| ✓ | BbhNavigate219CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT519 |
| ✓ | BbhNavigate220CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT366 |
| ✓ | BbhNavigate221CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT328 |
| ✓ | BbhNavigate222CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT255 |
| ✓ | BbhNavigate223CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT3 342 |
| ✓ | BbhNavigate224CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT715 |
| ✓ | BbhNavigate225CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT254 |
| ✓ | BbhNavigate226CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT274 |
| ✓ | BbhNavigate227CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT300 |
| ✓ | BbhNavigate228CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT165 |
| ✓ | BbhNavigate229CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT336 |
| ✓ | BbhNavigate230CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT225 |
| ✓ | BbhNavigate231CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT816 |
| ✓ | BbhNavigate232CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT304 |
| ✓ | BbhNavigate233CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT194 |
| ✓ | BbhNavigate234CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 309 |
| ✓ | BbhNavigate235CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT358 |
| ✓ | BbhNavigate236CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT313 |
| ✓ | BbhNavigate237CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT169 |
| ✓ | BbhNavigate238CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 007 |
| ✓ | BbhNavigate239CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT354 |
| ✓ | BbhNavigate240CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT189 |
| ✓ | BbhNavigate241CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT392 |
| ✓ | BbhNavigate242CAdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 256 |
Reasoning - BBH object counting (set 1)
31 passed1 failed |
96.9% |
IN31 770 OUT9 291 | |
| ✓ | BbhNavigate243CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 036 OUT1 828 |
| ✓ | BbhNavigate244CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT420 |
| ✓ | BbhNavigate245CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT215 |
| ✓ | BbhNavigate246CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT164 |
| ✓ | BbhNavigate247CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT693 |
| ✓ | BbhNavigate248CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 032 OUT580 |
| ✓ | BbhNavigate249CBdecide whether a navigation path returns to the start1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT871 |
| ✓ | BbhObjectCounting10CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT86 |
| ✓ | BbhObjectCounting11CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT663 |
| ✓ | BbhObjectCounting12CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT127 |
| ✓ | BbhObjectCounting13CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT87 |
| ✓ | BbhObjectCounting14CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT122 |
| ✓ | BbhObjectCounting15CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT198 |
| ✓ | BbhObjectCounting16CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT217 |
| ✓ | BbhObjectCounting17CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT70 |
| ✓ | BbhObjectCounting18CBI have a blackberry, a plum, and a banana. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT117 |
| ✓ | BbhObjectCounting19CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT146 |
| ✓ | BbhObjectCounting20CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT64 |
| ✓ | BbhObjectCounting21CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT151 |
| ✓ | BbhObjectCounting22CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT111 |
| × | BbhObjectCounting23CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "10")
Context: BbhObjectCounting23CB: expected 10, got [9] |
Failed | IN1 009 OUT400 |
| ✓ | BbhObjectCounting24CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT160 |
| ✓ | BbhObjectCounting25CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT186 |
| ✓ | BbhObjectCounting26CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT95 |
| ✓ | BbhObjectCounting27CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT185 |
| ✓ | BbhObjectCounting28CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT93 |
| ✓ | BbhObjectCounting29CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT153 |
| ✓ | BbhObjectCounting30CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT370 |
| ✓ | BbhObjectCounting31CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT118 |
| ✓ | BbhObjectCounting32CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT113 |
| ✓ | BbhObjectCounting8CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT363 |
| ✓ | BbhObjectCounting9CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT125 |
Reasoning - BBH object counting (set 2)
31 passed1 failed |
96.9% |
IN31 581 OUT5 989 | |
| ✓ | BbhObjectCounting33CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT98 |
| ✓ | BbhObjectCounting34CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT52 |
| ✓ | BbhObjectCounting35CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT86 |
| ✓ | BbhObjectCounting36CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT91 |
| ✓ | BbhObjectCounting37CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT111 |
| ✓ | BbhObjectCounting38CBI have a raspberry, a grape, and an orange. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT84 |
| ✓ | BbhObjectCounting39CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT246 |
| ✓ | BbhObjectCounting40CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT142 |
| ✓ | BbhObjectCounting41CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT55 |
| ✓ | BbhObjectCounting42CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT136 |
| ✓ | BbhObjectCounting43CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT128 |
| ✓ | BbhObjectCounting44CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT63 |
| ✓ | BbhObjectCounting45CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT84 |
| ✓ | BbhObjectCounting46CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT266 |
| ✓ | BbhObjectCounting47CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT335 |
| ✓ | BbhObjectCounting48CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT138 |
| × | BbhObjectCounting49CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "10")
Context: BbhObjectCounting49CB: expected 10, got [9] |
Failed | IN1 008 OUT281 |
| ✓ | BbhObjectCounting50CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT704 |
| ✓ | BbhObjectCounting51CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT398 |
| ✓ | BbhObjectCounting52CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT452 |
| ✓ | BbhObjectCounting53CBI have a frog, and a mouse. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT53 |
| ✓ | BbhObjectCounting54CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT110 |
| ✓ | BbhObjectCounting55CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT200 |
| ✓ | BbhObjectCounting56CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT266 |
| ✓ | BbhObjectCounting57CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT132 |
| ✓ | BbhObjectCounting58CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT177 |
| ✓ | BbhObjectCounting59CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT140 |
| ✓ | BbhObjectCounting60CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT404 |
| ✓ | BbhObjectCounting61CBI have a rabbit, a frog, and a snail. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT79 |
| ✓ | BbhObjectCounting62CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT238 |
| ✓ | BbhObjectCounting63CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT103 |
| ✓ | BbhObjectCounting64CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT137 |
Reasoning - BBH object counting (set 3)
31 passed1 failed |
96.9% |
IN31 651 OUT6 993 | |
| ✓ | BbhObjectCounting65CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT117 |
| ✓ | BbhObjectCounting66CBI have a grape, and a plum. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT63 |
| ✓ | BbhObjectCounting67CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT204 |
| ✓ | BbhObjectCounting68CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT49 |
| ✓ | BbhObjectCounting69CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT60 |
| ✓ | BbhObjectCounting70CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT368 |
| ✓ | BbhObjectCounting71CBI have a dog, a pig, and a snail. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT125 |
| ✓ | BbhObjectCounting72CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT155 |
| ✓ | BbhObjectCounting73CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT226 |
| ✓ | BbhObjectCounting74CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT291 |
| ✓ | BbhObjectCounting75CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT334 |
| ✓ | BbhObjectCounting76CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT282 |
| ✓ | BbhObjectCounting77CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT154 |
| ✓ | BbhObjectCounting78CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT370 |
| ✓ | BbhObjectCounting79CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT258 |
| ✓ | BbhObjectCounting80CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT132 |
| ✓ | BbhObjectCounting81CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT106 |
| ✓ | BbhObjectCounting82CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT159 |
| ✓ | BbhObjectCounting83CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT360 |
| ✓ | BbhObjectCounting84CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT163 |
| ✓ | BbhObjectCounting85CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT456 |
| ✓ | BbhObjectCounting86CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT303 |
| ✓ | BbhObjectCounting87CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT342 |
| × | BbhObjectCounting88CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "13")
Context: BbhObjectCounting88CB: expected 13, got [12] |
Failed | IN995 OUT156 |
| ✓ | BbhObjectCounting89CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT456 |
| ✓ | BbhObjectCounting90CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT186 |
| ✓ | BbhObjectCounting91CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT250 |
| ✓ | BbhObjectCounting92CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT288 |
| ✓ | BbhObjectCounting93CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT138 |
| ✓ | BbhObjectCounting94CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT164 |
| ✓ | BbhObjectCounting95CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT114 |
| ✓ | BbhObjectCounting96CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT164 |
Reasoning - BBH object counting (set 4)
31 passed |
100.0% |
IN30 522 OUT5 732 | |
| ✓ | BbhObjectCounting100CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT400 |
| ✓ | BbhObjectCounting101CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT92 |
| ✓ | BbhObjectCounting102CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT186 |
| ✓ | BbhObjectCounting103CBI have a table, and three cars. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT94 |
| ✓ | BbhObjectCounting104CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT243 |
| ✓ | BbhObjectCounting105CBI have a microwave, a bed, and a couch. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT70 |
| ✓ | BbhObjectCounting106CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT42 |
| ✓ | BbhObjectCounting107CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT263 |
| ✓ | BbhObjectCounting108CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT254 |
| ✓ | BbhObjectCounting109CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT216 |
| ✓ | BbhObjectCounting110CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT127 |
| ✓ | BbhObjectCounting111CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT55 |
| ✓ | BbhObjectCounting112CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT497 |
| ✓ | BbhObjectCounting113CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT146 |
| ✓ | BbhObjectCounting114CBI have a donkey, a goat, and a cow. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT56 |
| ✓ | BbhObjectCounting115CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT63 |
| ✓ | BbhObjectCounting116CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT136 |
| ✓ | BbhObjectCounting117CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT130 |
| ✓ | BbhObjectCounting118CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT310 |
| ✓ | BbhObjectCounting119CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT308 |
| ✓ | BbhObjectCounting120CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT213 |
| ✓ | BbhObjectCounting121CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT129 |
| ✓ | BbhObjectCounting122CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT349 |
| ✓ | BbhObjectCounting123CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT273 |
| ✓ | BbhObjectCounting124CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT114 |
| ✓ | BbhObjectCounting125CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT128 |
| ✓ | BbhObjectCounting126CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT88 |
| ✓ | BbhObjectCounting127CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT105 |
| ✓ | BbhObjectCounting97CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT373 |
| ✓ | BbhObjectCounting98CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT96 |
| ✓ | BbhObjectCounting99CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT176 |
Reasoning - BBH object counting (set 5)
31 passed |
100.0% |
IN30 467 OUT4 306 | |
| ✓ | BbhObjectCounting128CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT233 |
| ✓ | BbhObjectCounting129CBI have two grapes, and a plum. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT78 |
| ✓ | BbhObjectCounting130CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT80 |
| ✓ | BbhObjectCounting131CBI have a pig, and a donkey. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT58 |
| ✓ | BbhObjectCounting132CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT192 |
| ✓ | BbhObjectCounting133CBI have a microwave, and a bed. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT52 |
| ✓ | BbhObjectCounting134CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT77 |
| ✓ | BbhObjectCounting135CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT177 |
| ✓ | BbhObjectCounting136CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT71 |
| ✓ | BbhObjectCounting137CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT219 |
| ✓ | BbhObjectCounting138CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT64 |
| ✓ | BbhObjectCounting139CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT248 |
| ✓ | BbhObjectCounting140CBI have a chair, and two microwaves. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT111 |
| ✓ | BbhObjectCounting141CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT239 |
| ✓ | BbhObjectCounting142CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT163 |
| ✓ | BbhObjectCounting143CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT277 |
| ✓ | BbhObjectCounting144CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT218 |
| ✓ | BbhObjectCounting145CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT138 |
| ✓ | BbhObjectCounting146CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT89 |
| ✓ | BbhObjectCounting147CBI have a cat, and a fish. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT120 |
| ✓ | BbhObjectCounting148CBI have a head of broccoli, and a yam. How many vegetables do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT122 |
| ✓ | BbhObjectCounting149CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT101 |
| ✓ | BbhObjectCounting150CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT105 |
| ✓ | BbhObjectCounting151CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT67 |
| ✓ | BbhObjectCounting152CBI have four snails, and a cow. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT98 |
| ✓ | BbhObjectCounting153CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT198 |
| ✓ | BbhObjectCounting154CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT58 |
| ✓ | BbhObjectCounting155CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT125 |
| ✓ | BbhObjectCounting156CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT256 |
| ✓ | BbhObjectCounting157CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN976 OUT111 |
| ✓ | BbhObjectCounting158CBcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT161 |
Reasoning - BBH object counting (set 6)
29 passed3 failed |
90.6% |
IN31 544 OUT6 882 | |
| × | BbhObjectCounting159CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "8")
Context: BbhObjectCounting159CC: expected 8, got [6] |
Failed | IN989 OUT256 |
| ✓ | BbhObjectCounting160CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT102 |
| ✓ | BbhObjectCounting161CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT644 |
| ✓ | BbhObjectCounting162CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT160 |
| × | BbhObjectCounting163CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "13")
Context: BbhObjectCounting163CC: expected 13, got [12] |
Failed | IN1 001 OUT324 |
| ✓ | BbhObjectCounting164CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT100 |
| ✓ | BbhObjectCounting165CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT461 |
| ✓ | BbhObjectCounting166CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT181 |
| ✓ | BbhObjectCounting167CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT362 |
| ✓ | BbhObjectCounting168CCI have a toaster, a table, and a fridge. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT93 |
| ✓ | BbhObjectCounting169CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT157 |
| ✓ | BbhObjectCounting170CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT114 |
| ✓ | BbhObjectCounting171CCI have a garlic, and an onion. How many vegetables do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT181 |
| ✓ | BbhObjectCounting172CCI have a grape, a banana, and a blackberry. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT75 |
| × | BbhObjectCounting173CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "10")
Context: BbhObjectCounting173CC: expected 10, got [9] |
Failed | IN990 OUT289 |
| ✓ | BbhObjectCounting174CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT355 |
| ✓ | BbhObjectCounting175CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT332 |
| ✓ | BbhObjectCounting176CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT88 |
| ✓ | BbhObjectCounting177CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT135 |
| ✓ | BbhObjectCounting178CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT50 |
| ✓ | BbhObjectCounting179CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT309 |
| ✓ | BbhObjectCounting180CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT86 |
| ✓ | BbhObjectCounting181CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT166 |
| ✓ | BbhObjectCounting182CCI have a microwave, and a chair. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT53 |
| ✓ | BbhObjectCounting183CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT116 |
| ✓ | BbhObjectCounting184CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT172 |
| ✓ | BbhObjectCounting185CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT160 |
| ✓ | BbhObjectCounting186CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT562 |
| ✓ | BbhObjectCounting187CCI have a grape, an apple, and a banana. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT94 |
| ✓ | BbhObjectCounting188CCI have two blackberries, and a grape. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT73 |
| ✓ | BbhObjectCounting189CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT125 |
| ✓ | BbhObjectCounting190CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT507 |
Reasoning - BBH object counting (set 7)
29 passed3 failed |
90.6% |
IN31 621 OUT5 995 | |
| ✓ | BbhObjectCounting191CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT227 |
| ✓ | BbhObjectCounting192CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT89 |
| ✓ | BbhObjectCounting193CCI have a dog, a chicken, and four snails. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT114 |
| × | BbhObjectCounting194CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "13")
Context: BbhObjectCounting194CC: expected 13, got [12] |
Failed | IN997 OUT315 |
| ✓ | BbhObjectCounting195CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT306 |
| ✓ | BbhObjectCounting196CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT62 |
| ✓ | BbhObjectCounting197CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT146 |
| ✓ | BbhObjectCounting198CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT389 |
| ✓ | BbhObjectCounting199CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT318 |
| ✓ | BbhObjectCounting200CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT66 |
| ✓ | BbhObjectCounting201CCI have an apple, and two blackberries. How many fruits do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN972 OUT131 |
| ✓ | BbhObjectCounting202CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT119 |
| ✓ | BbhObjectCounting203CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT359 |
| ✓ | BbhObjectCounting204CCI have two lamps, and two microwaves. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN973 OUT104 |
| ✓ | BbhObjectCounting205CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT100 |
| × | BbhObjectCounting206CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "11")
Context: BbhObjectCounting206CC: expected 11, got [10] |
Failed | IN1 004 OUT356 |
| ✓ | BbhObjectCounting207CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT50 |
| ✓ | BbhObjectCounting208CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN989 OUT147 |
| ✓ | BbhObjectCounting209CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT387 |
| ✓ | BbhObjectCounting210CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN977 OUT104 |
| ✓ | BbhObjectCounting211CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT285 |
| ✓ | BbhObjectCounting212CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT199 |
| ✓ | BbhObjectCounting213CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT177 |
| ✓ | BbhObjectCounting214CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT382 |
| ✓ | BbhObjectCounting215CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT106 |
| × | BbhObjectCounting216CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: NumEquals(got, "11")
Context: BbhObjectCounting216CC: expected 11, got [12] |
Failed | IN996 OUT150 |
| ✓ | BbhObjectCounting217CCI have a toaster, a car, and a table. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT42 |
| ✓ | BbhObjectCounting218CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT319 |
| ✓ | BbhObjectCounting219CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT111 |
| ✓ | BbhObjectCounting220CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT112 |
| ✓ | BbhObjectCounting221CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT82 |
| ✓ | BbhObjectCounting222CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT141 |
Reasoning - BBH object counting (set 8)
27 passed |
100.0% |
IN26 689 OUT5 236 | |
| ✓ | BbhObjectCounting223CCI have an oven, a bed, and a toaster. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN974 OUT42 |
| ✓ | BbhObjectCounting224CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT411 |
| ✓ | BbhObjectCounting225CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT56 |
| ✓ | BbhObjectCounting226CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT328 |
| ✓ | BbhObjectCounting227CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN981 OUT125 |
| ✓ | BbhObjectCounting228CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT105 |
| ✓ | BbhObjectCounting229CCI have two cars, and an oven. How many objects do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN971 OUT48 |
| ✓ | BbhObjectCounting230CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT166 |
| ✓ | BbhObjectCounting231CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT307 |
| ✓ | BbhObjectCounting232CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT228 |
| ✓ | BbhObjectCounting233CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT135 |
| ✓ | BbhObjectCounting234CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT119 |
| ✓ | BbhObjectCounting235CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT261 |
| ✓ | BbhObjectCounting236CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT105 |
| ✓ | BbhObjectCounting237CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT515 |
| ✓ | BbhObjectCounting238CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT125 |
| ✓ | BbhObjectCounting239CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT335 |
| ✓ | BbhObjectCounting240CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT341 |
| ✓ | BbhObjectCounting241CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT89 |
| ✓ | BbhObjectCounting242CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT224 |
| ✓ | BbhObjectCounting243CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT129 |
| ✓ | BbhObjectCounting244CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT130 |
| ✓ | BbhObjectCounting245CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT185 |
| ✓ | BbhObjectCounting246CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT63 |
| ✓ | BbhObjectCounting247CCI have a fish, a snail, and two rabbits. How many animals do I have?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN975 OUT81 |
| ✓ | BbhObjectCounting248CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT97 |
| ✓ | BbhObjectCounting249CCcount objects of a given kind in a list1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT486 |
Reasoning - BBH penguins in a table (set 1)
32 passed1 failed |
97.0% |
IN38 044 OUT10 468 | |
| ✓ | BbhPenguinsInATable10CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT330 |
| ✓ | BbhPenguinsInATable11CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 203 OUT300 |
| ✓ | BbhPenguinsInATable12CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT316 |
| ✓ | BbhPenguinsInATable13CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 226 OUT126 |
| ✓ | BbhPenguinsInATable14CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT59 |
| ✓ | BbhPenguinsInATable15CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT532 |
| ✓ | BbhPenguinsInATable16CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 228 OUT296 |
| ✓ | BbhPenguinsInATable17CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT240 |
| ✓ | BbhPenguinsInATable18CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 137 OUT345 |
| ✓ | BbhPenguinsInATable19CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT558 |
| ✓ | BbhPenguinsInATable20CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT273 |
| ✓ | BbhPenguinsInATable21CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 138 OUT284 |
| ✓ | BbhPenguinsInATable22CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT198 |
| ✓ | BbhPenguinsInATable23CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT484 |
| ✓ | BbhPenguinsInATable24CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT250 |
| ✓ | BbhPenguinsInATable25CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 123 OUT356 |
| ✓ | BbhPenguinsInATable26CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 220 OUT236 |
| ✓ | BbhPenguinsInATable27CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT513 |
| ✓ | BbhPenguinsInATable28CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT203 |
| ✓ | BbhPenguinsInATable29CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 222 OUT196 |
| ✓ | BbhPenguinsInATable30CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 220 OUT411 |
| ✓ | BbhPenguinsInATable31CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 227 OUT167 |
| ✓ | BbhPenguinsInATable32CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 195 OUT170 |
| ✓ | BbhPenguinsInATable33CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 197 OUT83 |
| × | BbhPenguinsInATable34CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhPenguinsInATable34CC: expected (A), got [b]
Values:
got = "b" |
Failed | IN1 106 OUT1 338 |
| ✓ | BbhPenguinsInATable35CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT191 |
| ✓ | BbhPenguinsInATable36CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 137 OUT285 |
| ✓ | BbhPenguinsInATable37CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT215 |
| ✓ | BbhPenguinsInATable38CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 201 OUT213 |
| ✓ | BbhPenguinsInATable39CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT193 |
| ✓ | BbhPenguinsInATable40CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT541 |
| ✓ | BbhPenguinsInATable8CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT166 |
| ✓ | BbhPenguinsInATable9CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT400 |
Reasoning - BBH penguins in a table (set 2)
34 passed |
100.0% |
IN38 697 OUT7 861 | |
| ✓ | BbhPenguinsInATable41CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT249 |
| ✓ | BbhPenguinsInATable42CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT265 |
| ✓ | BbhPenguinsInATable43CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 226 OUT381 |
| ✓ | BbhPenguinsInATable44CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT239 |
| ✓ | BbhPenguinsInATable45CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT218 |
| ✓ | BbhPenguinsInATable46CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 219 OUT155 |
| ✓ | BbhPenguinsInATable47CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 200 OUT164 |
| ✓ | BbhPenguinsInATable48CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 138 OUT254 |
| ✓ | BbhPenguinsInATable49CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT249 |
| ✓ | BbhPenguinsInATable50CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 201 OUT203 |
| ✓ | BbhPenguinsInATable51CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT201 |
| ✓ | BbhPenguinsInATable52CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT194 |
| ✓ | BbhPenguinsInATable53CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT192 |
| ✓ | BbhPenguinsInATable54CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT261 |
| ✓ | BbhPenguinsInATable55CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT223 |
| ✓ | BbhPenguinsInATable56CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT256 |
| ✓ | BbhPenguinsInATable57CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT145 |
| ✓ | BbhPenguinsInATable58CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT151 |
| ✓ | BbhPenguinsInATable59CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT154 |
| ✓ | BbhPenguinsInATable60CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT184 |
| ✓ | BbhPenguinsInATable61CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT193 |
| ✓ | BbhPenguinsInATable62CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 119 OUT132 |
| ✓ | BbhPenguinsInATable63CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT195 |
| ✓ | BbhPenguinsInATable64CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 230 OUT426 |
| ✓ | BbhPenguinsInATable65CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT288 |
| ✓ | BbhPenguinsInATable66CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT182 |
| ✓ | BbhPenguinsInATable67CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 225 OUT148 |
| ✓ | BbhPenguinsInATable68CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT478 |
| ✓ | BbhPenguinsInATable69CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT177 |
| ✓ | BbhPenguinsInATable70CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT210 |
| ✓ | BbhPenguinsInATable71CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT440 |
| ✓ | BbhPenguinsInATable72CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT175 |
| ✓ | BbhPenguinsInATable73CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT163 |
| ✓ | BbhPenguinsInATable74CCanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 224 OUT316 |
Reasoning - BBH penguins in a table (set 3)
36 passed |
100.0% |
IN41 075 OUT10 457 | |
| ✓ | BbhPenguins100CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT452 |
| ✓ | BbhPenguins101CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT285 |
| ✓ | BbhPenguins102CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT359 |
| ✓ | BbhPenguins103CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT177 |
| ✓ | BbhPenguins104CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 199 OUT250 |
| ✓ | BbhPenguins105CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT402 |
| ✓ | BbhPenguins106CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT366 |
| ✓ | BbhPenguins107CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT355 |
| ✓ | BbhPenguins108CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT457 |
| ✓ | BbhPenguins109CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT138 |
| ✓ | BbhPenguins110CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT195 |
| ✓ | BbhPenguins75CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 224 OUT220 |
| ✓ | BbhPenguins76CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT437 |
| ✓ | BbhPenguins77CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 199 OUT224 |
| ✓ | BbhPenguins78CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT279 |
| ✓ | BbhPenguins79CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT386 |
| ✓ | BbhPenguins80CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 195 OUT194 |
| ✓ | BbhPenguins81CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT231 |
| ✓ | BbhPenguins82CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT420 |
| ✓ | BbhPenguins83CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT513 |
| ✓ | BbhPenguins84CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 210 OUT396 |
| ✓ | BbhPenguins85CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT388 |
| ✓ | BbhPenguins86CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT215 |
| ✓ | BbhPenguins87CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT349 |
| ✓ | BbhPenguins88CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT357 |
| ✓ | BbhPenguins89CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 225 OUT225 |
| ✓ | BbhPenguins90CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT169 |
| ✓ | BbhPenguins91CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT267 |
| ✓ | BbhPenguins92CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT303 |
| ✓ | BbhPenguins93CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT236 |
| ✓ | BbhPenguins94CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT254 |
| ✓ | BbhPenguins95CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT185 |
| ✓ | BbhPenguins96CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 231 OUT253 |
| ✓ | BbhPenguins97CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT190 |
| ✓ | BbhPenguins98CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 192 OUT122 |
| ✓ | BbhPenguins99CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT208 |
Reasoning - BBH penguins in a table (set 4)
35 passed |
100.0% |
IN39 978 OUT9 050 | |
| ✓ | BbhPenguins111CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT219 |
| ✓ | BbhPenguins112CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT136 |
| ✓ | BbhPenguins113CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 200 OUT208 |
| ✓ | BbhPenguins114CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 202 OUT229 |
| ✓ | BbhPenguins115CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT214 |
| ✓ | BbhPenguins116CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT227 |
| ✓ | BbhPenguins117CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 227 OUT252 |
| ✓ | BbhPenguins118CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT216 |
| ✓ | BbhPenguins119CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT175 |
| ✓ | BbhPenguins120CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT444 |
| ✓ | BbhPenguins121CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT391 |
| ✓ | BbhPenguins122CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 206 OUT219 |
| ✓ | BbhPenguins123CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT244 |
| ✓ | BbhPenguins124CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT184 |
| ✓ | BbhPenguins125CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 194 OUT154 |
| ✓ | BbhPenguins126CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT577 |
| ✓ | BbhPenguins127CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT463 |
| ✓ | BbhPenguins128CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 200 OUT198 |
| ✓ | BbhPenguins129CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT228 |
| ✓ | BbhPenguins130CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT284 |
| ✓ | BbhPenguins131CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT234 |
| ✓ | BbhPenguins132CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 232 OUT165 |
| ✓ | BbhPenguins133CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 205 OUT412 |
| ✓ | BbhPenguins134CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT229 |
| ✓ | BbhPenguins135CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT213 |
| ✓ | BbhPenguins136CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT163 |
| ✓ | BbhPenguins137CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT140 |
| ✓ | BbhPenguins138CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 100 OUT260 |
| ✓ | BbhPenguins139CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT321 |
| ✓ | BbhPenguins140CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 102 OUT387 |
| ✓ | BbhPenguins141CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 225 OUT307 |
| ✓ | BbhPenguins142CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT198 |
| ✓ | BbhPenguins143CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT288 |
| ✓ | BbhPenguins144CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT260 |
| ✓ | BbhPenguins145CDanswer a query over a table of penguins1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT211 |
Reasoning - BBH reasoning about colored objects (set 1)
33 passed |
100.0% |
IN35 257 OUT8 107 | |
| ✓ | BbhColoredObjects10CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT220 |
| ✓ | BbhColoredObjects11CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT271 |
| ✓ | BbhColoredObjects12CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT337 |
| ✓ | BbhColoredObjects13CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT194 |
| ✓ | BbhColoredObjects14CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT296 |
| ✓ | BbhColoredObjects15CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT135 |
| ✓ | BbhColoredObjects16CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT432 |
| ✓ | BbhColoredObjects17CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT205 |
| ✓ | BbhColoredObjects18CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT311 |
| ✓ | BbhColoredObjects19CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT272 |
| ✓ | BbhColoredObjects20CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 029 OUT258 |
| ✓ | BbhColoredObjects21CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT219 |
| ✓ | BbhColoredObjects22CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT207 |
| ✓ | BbhColoredObjects23CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 056 OUT91 |
| ✓ | BbhColoredObjects24CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT344 |
| ✓ | BbhColoredObjects25CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT485 |
| ✓ | BbhColoredObjects26CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT253 |
| ✓ | BbhColoredObjects27CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT269 |
| ✓ | BbhColoredObjects28CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT313 |
| ✓ | BbhColoredObjects29CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT259 |
| ✓ | BbhColoredObjects30CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT119 |
| ✓ | BbhColoredObjects31CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT140 |
| ✓ | BbhColoredObjects32CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT117 |
| ✓ | BbhColoredObjects33CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT215 |
| ✓ | BbhColoredObjects34CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT132 |
| ✓ | BbhColoredObjects35CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT266 |
| ✓ | BbhColoredObjects36CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT127 |
| ✓ | BbhColoredObjects37CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT296 |
| ✓ | BbhColoredObjects38CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT221 |
| ✓ | BbhColoredObjects39CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT276 |
| ✓ | BbhColoredObjects40CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT443 |
| ✓ | BbhColoredObjects8CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT158 |
| ✓ | BbhColoredObjects9CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT226 |
Reasoning - BBH reasoning about colored objects (set 2)
30 passed |
100.0% |
IN31 842 OUT7 003 | |
| ✓ | BbhColoredObjects41CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT241 |
| ✓ | BbhColoredObjects42CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT182 |
| ✓ | BbhColoredObjects43CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT56 |
| ✓ | BbhColoredObjects44CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT208 |
| ✓ | BbhColoredObjects45CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT155 |
| ✓ | BbhColoredObjects46CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT161 |
| ✓ | BbhColoredObjects47CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT87 |
| ✓ | BbhColoredObjects48CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT329 |
| ✓ | BbhColoredObjects49CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT424 |
| ✓ | BbhColoredObjects50CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT325 |
| ✓ | BbhColoredObjects51CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT569 |
| ✓ | BbhColoredObjects52CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT124 |
| ✓ | BbhColoredObjects53CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT264 |
| ✓ | BbhColoredObjects54CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT89 |
| ✓ | BbhColoredObjects55CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT303 |
| ✓ | BbhColoredObjects56CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT236 |
| ✓ | BbhColoredObjects57CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT297 |
| ✓ | BbhColoredObjects58CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT255 |
| ✓ | BbhColoredObjects59CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT356 |
| ✓ | BbhColoredObjects60CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT272 |
| ✓ | BbhColoredObjects61CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT182 |
| ✓ | BbhColoredObjects62CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT137 |
| ✓ | BbhColoredObjects63CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT297 |
| ✓ | BbhColoredObjects64CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT180 |
| ✓ | BbhColoredObjects65CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 039 OUT291 |
| ✓ | BbhColoredObjects66CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT294 |
| ✓ | BbhColoredObjects67CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT242 |
| ✓ | BbhColoredObjects68CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 034 OUT160 |
| ✓ | BbhColoredObjects69CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT206 |
| ✓ | BbhColoredObjects70CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT81 |
Reasoning - BBH reasoning about colored objects (set 3)
24 passed |
100.0% |
IN25 658 OUT6 687 | |
| ✓ | BbhColoredObjects71CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT168 |
| ✓ | BbhColoredObjects72CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT216 |
| ✓ | BbhColoredObjects73CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT343 |
| ✓ | BbhColoredObjects74CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT192 |
| ✓ | BbhColoredObjects75CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT1 358 |
| ✓ | BbhColoredObjects76CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 041 OUT282 |
| ✓ | BbhColoredObjects77CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT138 |
| ✓ | BbhColoredObjects78CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT336 |
| ✓ | BbhColoredObjects79CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT198 |
| ✓ | BbhColoredObjects80CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT232 |
| ✓ | BbhColoredObjects81CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT135 |
| ✓ | BbhColoredObjects82CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT170 |
| ✓ | BbhColoredObjects83CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT79 |
| ✓ | BbhColoredObjects84CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT178 |
| ✓ | BbhColoredObjects85CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT250 |
| ✓ | BbhColoredObjects86CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT527 |
| ✓ | BbhColoredObjects87CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT229 |
| ✓ | BbhColoredObjects88CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT182 |
| ✓ | BbhColoredObjects89CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 085 OUT220 |
| ✓ | BbhColoredObjects90CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT159 |
| ✓ | BbhColoredObjects91CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT226 |
| ✓ | BbhColoredObjects92CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT273 |
| ✓ | BbhColoredObjects93CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT295 |
| ✓ | BbhColoredObjects94CDreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT301 |
Reasoning - BBH reasoning about colored objects (set 4)
31 passed1 failed |
96.9% |
IN34 112 OUT7 895 | |
| ✓ | BbhReasoningAboutColoredObjects100CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT327 |
| ✓ | BbhReasoningAboutColoredObjects101CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT102 |
| ✓ | BbhReasoningAboutColoredObjects102CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT809 |
| ✓ | BbhReasoningAboutColoredObjects103CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT274 |
| ✓ | BbhReasoningAboutColoredObjects104CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT203 |
| ✓ | BbhReasoningAboutColoredObjects105CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT483 |
| ✓ | BbhReasoningAboutColoredObjects106CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT312 |
| ✓ | BbhReasoningAboutColoredObjects107CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT146 |
| ✓ | BbhReasoningAboutColoredObjects108CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT149 |
| ✓ | BbhReasoningAboutColoredObjects109CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT235 |
| ✓ | BbhReasoningAboutColoredObjects110CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT220 |
| ✓ | BbhReasoningAboutColoredObjects111CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT144 |
| ✓ | BbhReasoningAboutColoredObjects112CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT69 |
| ✓ | BbhReasoningAboutColoredObjects113CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT45 |
| ✓ | BbhReasoningAboutColoredObjects114CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT264 |
| ✓ | BbhReasoningAboutColoredObjects115CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT141 |
| ✓ | BbhReasoningAboutColoredObjects116CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT169 |
| ✓ | BbhReasoningAboutColoredObjects117CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT202 |
| ✓ | BbhReasoningAboutColoredObjects118CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT235 |
| × | BbhReasoningAboutColoredObjects119CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhReasoningAboutColoredObjects119CE: expected (D), got [h]
Values:
got = "h" |
Failed | IN1 098 OUT330 |
| ✓ | BbhReasoningAboutColoredObjects120CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT230 |
| ✓ | BbhReasoningAboutColoredObjects121CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT126 |
| ✓ | BbhReasoningAboutColoredObjects122CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT555 |
| ✓ | BbhReasoningAboutColoredObjects123CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT173 |
| ✓ | BbhReasoningAboutColoredObjects124CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT249 |
| ✓ | BbhReasoningAboutColoredObjects125CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT154 |
| ✓ | BbhReasoningAboutColoredObjects126CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT260 |
| ✓ | BbhReasoningAboutColoredObjects95CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT227 |
| ✓ | BbhReasoningAboutColoredObjects96CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT262 |
| ✓ | BbhReasoningAboutColoredObjects97CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 099 OUT457 |
| ✓ | BbhReasoningAboutColoredObjects98CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT180 |
| ✓ | BbhReasoningAboutColoredObjects99CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT163 |
Reasoning - BBH reasoning about colored objects (set 5)
32 passed |
100.0% |
IN33 832 OUT7 943 | |
| ✓ | BbhReasoningAboutColoredObjects127CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT315 |
| ✓ | BbhReasoningAboutColoredObjects128CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT173 |
| ✓ | BbhReasoningAboutColoredObjects129CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT184 |
| ✓ | BbhReasoningAboutColoredObjects130CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT159 |
| ✓ | BbhReasoningAboutColoredObjects131CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT330 |
| ✓ | BbhReasoningAboutColoredObjects132CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT505 |
| ✓ | BbhReasoningAboutColoredObjects133CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT353 |
| ✓ | BbhReasoningAboutColoredObjects134CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT210 |
| ✓ | BbhReasoningAboutColoredObjects135CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT76 |
| ✓ | BbhReasoningAboutColoredObjects136CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT268 |
| ✓ | BbhReasoningAboutColoredObjects137CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT116 |
| ✓ | BbhReasoningAboutColoredObjects138CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT399 |
| ✓ | BbhReasoningAboutColoredObjects139CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT123 |
| ✓ | BbhReasoningAboutColoredObjects140CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT231 |
| ✓ | BbhReasoningAboutColoredObjects141CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT104 |
| ✓ | BbhReasoningAboutColoredObjects142CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT66 |
| ✓ | BbhReasoningAboutColoredObjects143CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT206 |
| ✓ | BbhReasoningAboutColoredObjects144CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 043 OUT251 |
| ✓ | BbhReasoningAboutColoredObjects145CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT443 |
| ✓ | BbhReasoningAboutColoredObjects146CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT147 |
| ✓ | BbhReasoningAboutColoredObjects147CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 035 OUT286 |
| ✓ | BbhReasoningAboutColoredObjects148CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 098 OUT293 |
| ✓ | BbhReasoningAboutColoredObjects149CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 027 OUT273 |
| ✓ | BbhReasoningAboutColoredObjects150CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT394 |
| ✓ | BbhReasoningAboutColoredObjects151CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 121 OUT487 |
| ✓ | BbhReasoningAboutColoredObjects152CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT83 |
| ✓ | BbhReasoningAboutColoredObjects153CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT255 |
| ✓ | BbhReasoningAboutColoredObjects154CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT268 |
| ✓ | BbhReasoningAboutColoredObjects155CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT290 |
| ✓ | BbhReasoningAboutColoredObjects156CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT344 |
| ✓ | BbhReasoningAboutColoredObjects157CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT238 |
| ✓ | BbhReasoningAboutColoredObjects158CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT73 |
Reasoning - BBH reasoning about colored objects (set 6)
32 passed |
100.0% |
IN34 075 OUT6 805 | |
| ✓ | BbhReasoningAboutColoredObjects159CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT71 |
| ✓ | BbhReasoningAboutColoredObjects160CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT310 |
| ✓ | BbhReasoningAboutColoredObjects161CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT149 |
| ✓ | BbhReasoningAboutColoredObjects162CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 069 OUT193 |
| ✓ | BbhReasoningAboutColoredObjects163CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT115 |
| ✓ | BbhReasoningAboutColoredObjects164CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT187 |
| ✓ | BbhReasoningAboutColoredObjects165CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT184 |
| ✓ | BbhReasoningAboutColoredObjects166CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT162 |
| ✓ | BbhReasoningAboutColoredObjects167CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT340 |
| ✓ | BbhReasoningAboutColoredObjects168CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 083 OUT343 |
| ✓ | BbhReasoningAboutColoredObjects169CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT134 |
| ✓ | BbhReasoningAboutColoredObjects170CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT141 |
| ✓ | BbhReasoningAboutColoredObjects171CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT286 |
| ✓ | BbhReasoningAboutColoredObjects172CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT153 |
| ✓ | BbhReasoningAboutColoredObjects173CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT106 |
| ✓ | BbhReasoningAboutColoredObjects174CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 036 OUT329 |
| ✓ | BbhReasoningAboutColoredObjects175CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 040 OUT364 |
| ✓ | BbhReasoningAboutColoredObjects176CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT196 |
| ✓ | BbhReasoningAboutColoredObjects177CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT182 |
| ✓ | BbhReasoningAboutColoredObjects178CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT101 |
| ✓ | BbhReasoningAboutColoredObjects179CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT196 |
| ✓ | BbhReasoningAboutColoredObjects180CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 088 OUT115 |
| ✓ | BbhReasoningAboutColoredObjects181CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT137 |
| ✓ | BbhReasoningAboutColoredObjects182CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 094 OUT408 |
| ✓ | BbhReasoningAboutColoredObjects183CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT102 |
| ✓ | BbhReasoningAboutColoredObjects184CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT323 |
| ✓ | BbhReasoningAboutColoredObjects185CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT225 |
| ✓ | BbhReasoningAboutColoredObjects186CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 077 OUT189 |
| ✓ | BbhReasoningAboutColoredObjects187CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT358 |
| ✓ | BbhReasoningAboutColoredObjects188CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT287 |
| ✓ | BbhReasoningAboutColoredObjects189CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT125 |
| ✓ | BbhReasoningAboutColoredObjects190CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT294 |
Reasoning - BBH reasoning about colored objects (set 7)
32 passed |
100.0% |
IN34 010 OUT7 623 | |
| ✓ | BbhReasoningAboutColoredObjects191CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 080 OUT272 |
| ✓ | BbhReasoningAboutColoredObjects192CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT327 |
| ✓ | BbhReasoningAboutColoredObjects193CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT135 |
| ✓ | BbhReasoningAboutColoredObjects194CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT114 |
| ✓ | BbhReasoningAboutColoredObjects195CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT104 |
| ✓ | BbhReasoningAboutColoredObjects196CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT365 |
| ✓ | BbhReasoningAboutColoredObjects197CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT324 |
| ✓ | BbhReasoningAboutColoredObjects198CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 095 OUT134 |
| ✓ | BbhReasoningAboutColoredObjects199CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT534 |
| ✓ | BbhReasoningAboutColoredObjects200CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT517 |
| ✓ | BbhReasoningAboutColoredObjects201CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT143 |
| ✓ | BbhReasoningAboutColoredObjects202CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT133 |
| ✓ | BbhReasoningAboutColoredObjects203CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 055 OUT147 |
| ✓ | BbhReasoningAboutColoredObjects204CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT291 |
| ✓ | BbhReasoningAboutColoredObjects205CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT220 |
| ✓ | BbhReasoningAboutColoredObjects206CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 064 OUT170 |
| ✓ | BbhReasoningAboutColoredObjects207CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT119 |
| ✓ | BbhReasoningAboutColoredObjects208CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT200 |
| ✓ | BbhReasoningAboutColoredObjects209CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT335 |
| ✓ | BbhReasoningAboutColoredObjects210CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT191 |
| ✓ | BbhReasoningAboutColoredObjects211CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT60 |
| ✓ | BbhReasoningAboutColoredObjects212CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 028 OUT257 |
| ✓ | BbhReasoningAboutColoredObjects213CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT334 |
| ✓ | BbhReasoningAboutColoredObjects214CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 096 OUT141 |
| ✓ | BbhReasoningAboutColoredObjects215CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 089 OUT239 |
| ✓ | BbhReasoningAboutColoredObjects216CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT121 |
| ✓ | BbhReasoningAboutColoredObjects217CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 071 OUT158 |
| ✓ | BbhReasoningAboutColoredObjects218CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT223 |
| ✓ | BbhReasoningAboutColoredObjects219CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT102 |
| ✓ | BbhReasoningAboutColoredObjects220CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT663 |
| ✓ | BbhReasoningAboutColoredObjects221CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 045 OUT297 |
| ✓ | BbhReasoningAboutColoredObjects222CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 086 OUT253 |
Reasoning - BBH reasoning about colored objects (set 8)
29 passed1 failed |
96.7% |
IN31 734 OUT8 553 | |
| ✓ | BbhReasoningAboutColoredObjects223CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 091 OUT380 |
| ✓ | BbhReasoningAboutColoredObjects224CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT223 |
| ✓ | BbhReasoningAboutColoredObjects225CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT58 |
| ✓ | BbhReasoningAboutColoredObjects226CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT211 |
| ✓ | BbhReasoningAboutColoredObjects227CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 051 OUT178 |
| ✓ | BbhReasoningAboutColoredObjects228CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 046 OUT240 |
| ✓ | BbhReasoningAboutColoredObjects229CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT106 |
| ✓ | BbhReasoningAboutColoredObjects230CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT117 |
| ✓ | BbhReasoningAboutColoredObjects231CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT101 |
| ✓ | BbhReasoningAboutColoredObjects232CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 052 OUT420 |
| ✓ | BbhReasoningAboutColoredObjects233CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 081 OUT281 |
| ✓ | BbhReasoningAboutColoredObjects234CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 047 OUT300 |
| ✓ | BbhReasoningAboutColoredObjects235CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT258 |
| ✓ | BbhReasoningAboutColoredObjects236CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 092 OUT372 |
| ✓ | BbhReasoningAboutColoredObjects237CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT127 |
| ✓ | BbhReasoningAboutColoredObjects238CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT331 |
| ✓ | BbhReasoningAboutColoredObjects239CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 079 OUT187 |
| ✓ | BbhReasoningAboutColoredObjects240CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT139 |
| ✓ | BbhReasoningAboutColoredObjects241CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 062 OUT136 |
| ✓ | BbhReasoningAboutColoredObjects242CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 097 OUT140 |
| ✓ | BbhReasoningAboutColoredObjects243CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 090 OUT402 |
| ✓ | BbhReasoningAboutColoredObjects244CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT305 |
| ✓ | BbhReasoningAboutColoredObjects245CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 087 OUT220 |
| ✓ | BbhReasoningAboutColoredObjects246CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 048 OUT243 |
| ✓ | BbhReasoningAboutColoredObjects247CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT140 |
| ✓ | BbhReasoningAboutColoredObjects248CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 093 OUT128 |
| ✓ | BbhReasoningAboutColoredObjects249CEreason about colored objects' positions/colors1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 075 OUT240 |
| × | BbhRuinNames10CEpick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames10CE: expected (A), got [d]
Values:
got = "d" |
Failed | IN998 OUT854 |
| ✓ | BbhRuinNames8CEpick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT908 |
| ✓ | BbhRuinNames9CEpick the humorous one-edit change of a name1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT808 |
Reasoning - BBH ruin names (humorous edits) (set 1)
24 passed8 failed |
75.0% |
IN32 164 OUT26 270 | |
| ✓ | BbhRuinNames11CFWhich of the following is a humorous edit of this artist or movie name: 'the who!'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT440 |
| ✓ | BbhRuinNames12CFWhich of the following is a humorous edit of this artist or movie name: 'braveheart'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT371 |
| × | BbhRuinNames13CFWhich of the following is a humorous edit of this artist or movie name: 'the three stooges'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames13CF: expected (D), got [b]
Values:
got = "b" |
Failed | IN1 008 OUT1 662 |
| ✓ | BbhRuinNames14CFWhich of the following is a humorous edit of this artist or movie name: 'the deer hunter'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT339 |
| × | BbhRuinNames15CFWhich of the following is a humorous edit of this artist or movie name: 'boyz 2 men'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames15CF: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 013 OUT1 815 |
| ✓ | BbhRuinNames16CFWhich of the following is a humorous edit of this artist or movie name: 'the beastie boys'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT397 |
| × | BbhRuinNames17CFWhich of the following is a humorous edit of this artist or movie name: 'joy division'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames17CF: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 000 OUT1 303 |
| ✓ | BbhRuinNames18CFWhich of the following is a humorous edit of this artist or movie name: 'herbie hancock'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 240 |
| ✓ | BbhRuinNames19CFWhich of the following is a humorous edit of this artist or movie name: 'bon iver'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT979 |
| × | BbhRuinNames20CFWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames20CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN1 000 OUT316 |
| ✓ | BbhRuinNames21CFWhich of the following is a humorous edit of this artist or movie name: 'cinderella'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT351 |
| ✓ | BbhRuinNames22CFWhich of the following is a humorous edit of this artist or movie name: 'the italian job'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 311 |
| ✓ | BbhRuinNames23CFWhich of the following is a humorous edit of this artist or movie name: 'a few good men'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT340 |
| ✓ | BbhRuinNames24CFWhich of the following is a humorous edit of this artist or movie name: 'raiders of the lost ark'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT498 |
| ✓ | BbhRuinNames25CFWhich of the following is a humorous edit of this artist or movie name: 'little shop of horrors'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT496 |
| ✓ | BbhRuinNames26CFWhich of the following is a humorous edit of this artist or movie name: 'free willy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT808 |
| ✓ | BbhRuinNames27CFWhich of the following is a humorous edit of this artist or movie name: 'pixies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT391 |
| ✓ | BbhRuinNames28CFWhich of the following is a humorous edit of this artist or movie name: 'the princess and the frog'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT280 |
| ✓ | BbhRuinNames29CFWhich of the following is a humorous edit of this artist or movie name: 'foo fighters'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT331 |
| ✓ | BbhRuinNames30CFWhich of the following is a humorous edit of this artist or movie name: 'the godfather'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT586 |
| ✓ | BbhRuinNames31CFWhich of the following is a humorous edit of this artist or movie name: 'pantera'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 194 |
| × | BbhRuinNames32CFWhich of the following is a humorous edit of this artist or movie name: 'top gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames32CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN997 OUT1 023 |
| ✓ | BbhRuinNames33CFWhich of the following is a humorous edit of this artist or movie name: 'die another day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT299 |
| ✓ | BbhRuinNames34CFWhich of the following is a humorous edit of this artist or movie name: 'the moody blues'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT890 |
| ✓ | BbhRuinNames35CFWhich of the following is a humorous edit of this artist or movie name: 'fall out boy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT389 |
| × | BbhRuinNames36CFWhich of the following is a humorous edit of this artist or movie name: 'love story'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames36CF: expected (A), got [c]
Values:
got = "c" |
Failed | IN1 000 OUT1 055 |
| ✓ | BbhRuinNames37CFWhich of the following is a humorous edit of this artist or movie name: 'whitney houston'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT1 192 |
| ✓ | BbhRuinNames38CFWhich of the following is a humorous edit of this artist or movie name: 'bridge of spies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT384 |
| × | BbhRuinNames39CFWhich of the following is a humorous edit of this artist or movie name: 'breaking benjamin'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames39CF: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 007 OUT349 |
| × | BbhRuinNames40CFWhich of the following is a humorous edit of this artist or movie name: 'frank kappa'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames40CF: expected (C), got [d]
Values:
got = "d" |
Failed | IN1 001 OUT2 951 |
| ✓ | BbhRuinNames41CFWhich of the following is a humorous edit of this artist or movie name: 'jurassic park'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT780 |
| ✓ | BbhRuinNames42CFWhich of the following is a humorous edit of this artist or movie name: 'the band'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 510 |
Reasoning - BBH ruin names (humorous edits) (set 2)
27 passed5 failed |
84.4% |
IN32 200 OUT24 923 | |
| ✓ | BbhRuinNames43CFWhich of the following is a humorous edit of this artist or movie name: 'creedence clearwater revival'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT380 |
| × | BbhRuinNames44CFWhich of the following is a humorous edit of this artist or movie name: 'block 57'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames44CF: expected (D), got [a]
Values:
got = "a" |
Failed | IN1 010 OUT1 420 |
| ✓ | BbhRuinNames45CFWhich of the following is a humorous edit of this artist or movie name: 'get shorty'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT483 |
| × | BbhRuinNames46CFWhich of the following is a humorous edit of this artist or movie name: 'radiohead'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames46CF: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 000 OUT654 |
| ✓ | BbhRuinNames47CFWhich of the following is a humorous edit of this artist or movie name: 'mean girls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT371 |
| ✓ | BbhRuinNames48CFWhich of the following is a humorous edit of this artist or movie name: 'seven brides for seven brothers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT329 |
| ✓ | BbhRuinNames49CFWhich of the following is a humorous edit of this artist or movie name: 'the green mile'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 307 |
| ✓ | BbhRuinNames50CFWhich of the following is a humorous edit of this artist or movie name: 'cat on a hot tin roof'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT380 |
| ✓ | BbhRuinNames51CFWhich of the following is a humorous edit of this artist or movie name: 'new kids on the block'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT739 |
| ✓ | BbhRuinNames52CFWhich of the following is a humorous edit of this artist or movie name: 'superclunk'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 576 |
| ✓ | BbhRuinNames53CFWhich of the following is a humorous edit of this artist or movie name: 'my big fat greek wedding'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT346 |
| ✓ | BbhRuinNames54CFWhich of the following is a humorous edit of this artist or movie name: 'snakes on a plane'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT495 |
| × | BbhRuinNames55CFWhich of the following is a humorous edit of this artist or movie name: 'rocky'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames55CF: expected (A), got [d]
Values:
got = "d" |
Failed | IN997 OUT879 |
| ✓ | BbhRuinNames56CFWhich of the following is a humorous edit of this artist or movie name: 'pink floyd'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT481 |
| ✓ | BbhRuinNames57CFWhich of the following is a humorous edit of this artist or movie name: 'bad religion'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT531 |
| × | BbhRuinNames58CFWhich of the following is a humorous edit of this artist or movie name: 'the police'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames58CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN1 000 OUT711 |
| ✓ | BbhRuinNames59CFWhich of the following is a humorous edit of this artist or movie name: 'ice cube'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT283 |
| ✓ | BbhRuinNames60CFWhich of the following is a humorous edit of this artist or movie name: 'new order'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT889 |
| ✓ | BbhRuinNames61CFWhich of the following is a humorous edit of this artist or movie name: 'dolly dots'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 297 |
| ✓ | BbhRuinNames62CFWhich of the following is a humorous edit of this artist or movie name: 'die another day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT451 |
| ✓ | BbhRuinNames63CFWhich of the following is a humorous edit of this artist or movie name: 'mad max'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT639 |
| ✓ | BbhRuinNames64CFWhich of the following is a humorous edit of this artist or movie name: '8 mile'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT420 |
| ✓ | BbhRuinNames65CFWhich of the following is a humorous edit of this artist or movie name: 'i spit on your grave'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT459 |
| ✓ | BbhRuinNames66CFWhich of the following is a humorous edit of this artist or movie name: 'planet of the apes'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT306 |
| ✓ | BbhRuinNames67CFWhich of the following is a humorous edit of this artist or movie name: 'the rolling stones'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT305 |
| ✓ | BbhRuinNames68CFWhich of the following is a humorous edit of this artist or movie name: 'reservoir dogs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT320 |
| ✓ | BbhRuinNames69CFWhich of the following is a humorous edit of this artist or movie name: 'the men who stare at goats'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT1 291 |
| ✓ | BbhRuinNames70CFWhich of the following is a humorous edit of this artist or movie name: 'bridge of spies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT354 |
| ✓ | BbhRuinNames71CFWhich of the following is a humorous edit of this artist or movie name: 'last action hero'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT741 |
| ✓ | BbhRuinNames72CFWhich of the following is a humorous edit of this artist or movie name: 'the godfather'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT369 |
| ✓ | BbhRuinNames73CFWhich of the following is a humorous edit of this artist or movie name: 'the full monty'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT763 |
| × | BbhRuinNames74CFWhich of the following is a humorous edit of this artist or movie name: 'fast and furious'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames74CF: expected (B), got [c]
Values:
got = "c" |
Failed | IN1 005 OUT4 954 |
Reasoning - BBH ruin names (humorous edits) (set 3)
27 passed5 failed |
84.4% |
IN32 223 OUT19 818 | |
| ✓ | BbhRuinNames100CFWhich of the following is a humorous edit of this artist or movie name: 'the battle of britain'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT442 |
| × | BbhRuinNames101CFWhich of the following is a humorous edit of this artist or movie name: 'dead man walking'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames101CF: expected (A), got [d]
Values:
got = "d" |
Failed | IN1 005 OUT387 |
| ✓ | BbhRuinNames102CFWhich of the following is a humorous edit of this artist or movie name: 'suicide squad'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT286 |
| × | BbhRuinNames103CFWhich of the following is a humorous edit of this artist or movie name: 'the dark knight'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames103CF: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 006 OUT575 |
| ✓ | BbhRuinNames104CFWhich of the following is a humorous edit of this artist or movie name: 'sleepless in seattle'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT557 |
| ✓ | BbhRuinNames105CFWhich of the following is a humorous edit of this artist or movie name: 'pink floyd'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 125 |
| ✓ | BbhRuinNames106CFWhich of the following is a humorous edit of this artist or movie name: 'the man with the golden gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT864 |
| ✓ | BbhRuinNames75CFWhich of the following is a humorous edit of this artist or movie name: 'skinny puppy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT577 |
| ✓ | BbhRuinNames76CFWhich of the following is a humorous edit of this artist or movie name: 'depeche mode'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT758 |
| ✓ | BbhRuinNames77CFWhich of the following is a humorous edit of this artist or movie name: 'sly and the family stone'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT391 |
| ✓ | BbhRuinNames78CFWhich of the following is a humorous edit of this artist or movie name: 'the departed'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT340 |
| ✓ | BbhRuinNames79CFWhich of the following is a humorous edit of this artist or movie name: 'blade runner'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT339 |
| ✓ | BbhRuinNames80CFWhich of the following is a humorous edit of this artist or movie name: 'the avengers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT828 |
| ✓ | BbhRuinNames81CFWhich of the following is a humorous edit of this artist or movie name: 'die hard'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT944 |
| ✓ | BbhRuinNames82CFWhich of the following is a humorous edit of this artist or movie name: 'weezer'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT269 |
| ✓ | BbhRuinNames83CFWhich of the following is a humorous edit of this artist or movie name: 'the longest day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT380 |
| × | BbhRuinNames84CFWhich of the following is a humorous edit of this artist or movie name: 'babs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames84CF: expected (A), got [d]
Values:
got = "d" |
Failed | IN996 OUT1 661 |
| ✓ | BbhRuinNames85CFWhich of the following is a humorous edit of this artist or movie name: 'moonlight'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT686 |
| ✓ | BbhRuinNames86CFWhich of the following is a humorous edit of this artist or movie name: 'animal farm'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT294 |
| ✓ | BbhRuinNames87CFWhich of the following is a humorous edit of this artist or movie name: 'seven brides for seven brothers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT319 |
| ✓ | BbhRuinNames88CFWhich of the following is a humorous edit of this artist or movie name: 'kiss'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT482 |
| ✓ | BbhRuinNames89CFWhich of the following is a humorous edit of this artist or movie name: 'the police'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT379 |
| ✓ | BbhRuinNames90CFWhich of the following is a humorous edit of this artist or movie name: 'en vogue'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 113 |
| ✓ | BbhRuinNames91CFWhich of the following is a humorous edit of this artist or movie name: 'top gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT442 |
| ✓ | BbhRuinNames92CFWhich of the following is a humorous edit of this artist or movie name: 'the girl with all the gifts'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 021 OUT360 |
| ✓ | BbhRuinNames93CFWhich of the following is a humorous edit of this artist or movie name: 'the sisters of mercy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT393 |
| × | BbhRuinNames94CFWhich of the following is a humorous edit of this artist or movie name: 'the jam'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames94CF: expected (C), got [d]
Values:
got = "d" |
Failed | IN998 OUT724 |
| ✓ | BbhRuinNames95CFWhich of the following is a humorous edit of this artist or movie name: 'gone with the wind'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT927 |
| ✓ | BbhRuinNames96CFWhich of the following is a humorous edit of this artist or movie name: 'gone with the wind'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT369 |
| ✓ | BbhRuinNames97CFWhich of the following is a humorous edit of this artist or movie name: 'dream theater'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT829 |
| ✓ | BbhRuinNames98CFWhich of the following is a humorous edit of this artist or movie name: 'belle and sebastian'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT499 |
| × | BbhRuinNames99CFWhich of the following is a humorous edit of this artist or movie name: 'earth, wind, & fire'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("dearth, wind, & fire")
Context: BbhRuinNames99CF: expected dearth, wind, & fire, got [i]
Values:
got = "i" |
Failed | IN1 033 OUT1 279 |
Reasoning - BBH ruin names (humorous edits) (set 4)
27 passed5 failed |
84.4% |
IN32 207 OUT25 786 | |
| × | BbhRuinNames107CFWhich of the following is a humorous edit of this artist or movie name: 'the mash'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames107CF: expected (D), got [a]
Values:
got = "a" |
Failed | IN1 000 OUT1 006 |
| ✓ | BbhRuinNames108CFWhich of the following is a humorous edit of this artist or movie name: 'missy elliott'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT345 |
| ✓ | BbhRuinNames109CFWhich of the following is a humorous edit of this artist or movie name: 'sleater-kinney'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT467 |
| ✓ | BbhRuinNames110CFWhich of the following is a humorous edit of this artist or movie name: 'dirty dancing'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 250 |
| ✓ | BbhRuinNames111CFWhich of the following is a humorous edit of this artist or movie name: 'jay and silent bob strike back'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT1 607 |
| × | BbhRuinNames112CFWhich of the following is a humorous edit of this artist or movie name: 'a bridge too far'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames112CF: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 009 OUT982 |
| ✓ | BbhRuinNames113CFWhich of the following is a humorous edit of this artist or movie name: 'the beatles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT565 |
| × | BbhRuinNames114CFWhich of the following is a humorous edit of this artist or movie name: 'night club'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames114CF: expected (A), got [d]
Values:
got = "d" |
Failed | IN1 002 OUT1 863 |
| ✓ | BbhRuinNames115CFWhich of the following is a humorous edit of this artist or movie name: 'no time to die'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT278 |
| × | BbhRuinNames116CFWhich of the following is a humorous edit of this artist or movie name: 'batman begins'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames116CF: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 003 OUT378 |
| ✓ | BbhRuinNames117CFWhich of the following is a humorous edit of this artist or movie name: 'gold finger'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 114 |
| ✓ | BbhRuinNames118CFWhich of the following is a humorous edit of this artist or movie name: 'calamity jane'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT887 |
| ✓ | BbhRuinNames119CFWhich of the following is a humorous edit of this artist or movie name: 'saving private ryan'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT770 |
| ✓ | BbhRuinNames120CFWhich of the following is a humorous edit of this artist or movie name: 'the cult'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 442 |
| ✓ | BbhRuinNames121CFWhich of the following is a humorous edit of this artist or movie name: 'johnny got his gun'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT448 |
| × | BbhRuinNames122CFWhich of the following is a humorous edit of this artist or movie name: 'counting crows'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames122CF: expected (D), got [a]
Values:
got = "a" |
Failed | IN1 004 OUT846 |
| ✓ | BbhRuinNames123CFWhich of the following is a humorous edit of this artist or movie name: 'lawrence of arabia'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT509 |
| ✓ | BbhRuinNames124CFWhich of the following is a humorous edit of this artist or movie name: 'jungle book'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT515 |
| ✓ | BbhRuinNames125CFWhich of the following is a humorous edit of this artist or movie name: 'paint your wagon'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT505 |
| ✓ | BbhRuinNames126CFWhich of the following is a humorous edit of this artist or movie name: 'blade'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 128 |
| ✓ | BbhRuinNames127CFWhich of the following is a humorous edit of this artist or movie name: 'the sisterhood of the traveling pants'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT430 |
| ✓ | BbhRuinNames128CFWhich of the following is a humorous edit of this artist or movie name: 'green day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 465 |
| ✓ | BbhRuinNames129CFWhich of the following is a humorous edit of this artist or movie name: 'the sting'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT374 |
| ✓ | BbhRuinNames130CFWhich of the following is a humorous edit of this artist or movie name: 'the artist'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT280 |
| ✓ | BbhRuinNames131CFWhich of the following is a humorous edit of this artist or movie name: 'the hope'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT935 |
| ✓ | BbhRuinNames132CFWhich of the following is a humorous edit of this artist or movie name: 'blade runner'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT2 298 |
| ✓ | BbhRuinNames133CFWhich of the following is a humorous edit of this artist or movie name: 'a view to a kill'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT341 |
| ✓ | BbhRuinNames134CFWhich of the following is a humorous edit of this artist or movie name: 'hope floats'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT386 |
| ✓ | BbhRuinNames135CFWhich of the following is a humorous edit of this artist or movie name: 'the apartment'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 238 |
| ✓ | BbhRuinNames136CFWhich of the following is a humorous edit of this artist or movie name: 'schindler's list'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT472 |
| ✓ | BbhRuinNames137CFWhich of the following is a humorous edit of this artist or movie name: 'beauty and the beast'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT331 |
| ✓ | BbhRuinNames138CFWhich of the following is a humorous edit of this artist or movie name: 'news of the world'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT331 |
Reasoning - BBH ruin names (humorous edits) (set 5)
25 passed5 failed |
83.3% |
IN30 199 OUT22 694 | |
| ✓ | BbhRuinNames139CFWhich of the following is a humorous edit of this artist or movie name: 'the guns of navarone'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT390 |
| ✓ | BbhRuinNames140CFWhich of the following is a humorous edit of this artist or movie name: 'the beatles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT287 |
| ✓ | BbhRuinNames141CFWhich of the following is a humorous edit of this artist or movie name: 'the godfather'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT358 |
| ✓ | BbhRuinNames142CFWhich of the following is a humorous edit of this artist or movie name: 'hank williams'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT891 |
| ✓ | BbhRuinNames143CFWhich of the following is a humorous edit of this artist or movie name: 'black sabbath'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 163 |
| × | BbhRuinNames144CFWhich of the following is a humorous edit of this artist or movie name: 'rita, sue and bob too'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("rita, sue and bob poo")
Context: BbhRuinNames144CF: expected rita, sue and bob poo, got [d]
Values:
got = "d" |
Failed | IN1 027 OUT353 |
| ✓ | BbhRuinNames145CFWhich of the following is a humorous edit of this artist or movie name: 'jethro tull'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT366 |
| ✓ | BbhRuinNames146CFWhich of the following is a humorous edit of this artist or movie name: 'the sound of music'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT327 |
| ✓ | BbhRuinNames147CFWhich of the following is a humorous edit of this artist or movie name: 'dream theater'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT945 |
| ✓ | BbhRuinNames148CFWhich of the following is a humorous edit of this artist or movie name: 'my fair lady'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 011 |
| × | BbhRuinNames149CFWhich of the following is a humorous edit of this artist or movie name: 'offspring'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames149CF: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 000 OUT2 251 |
| × | BbhRuinNames150CFWhich of the following is a humorous edit of this artist or movie name: 'the longest day'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames150CF: expected (A), got [b]
Values:
got = "b" |
Failed | IN1 006 OUT1 772 |
| ✓ | BbhRuinNames151CFWhich of the following is a humorous edit of this artist or movie name: 'apocalypse now'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT426 |
| ✓ | BbhRuinNames152CFWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT1 992 |
| ✓ | BbhRuinNames153CFWhich of the following is a humorous edit of this artist or movie name: 'big country'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT890 |
| ✓ | BbhRuinNames154CFWhich of the following is a humorous edit of this artist or movie name: '12 angry men'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT278 |
| ✓ | BbhRuinNames155CFWhich of the following is a humorous edit of this artist or movie name: 'in search of a midnight kiss'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 023 OUT325 |
| ✓ | BbhRuinNames156CFWhich of the following is a humorous edit of this artist or movie name: 'men in black'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT448 |
| ✓ | BbhRuinNames157CFWhich of the following is a humorous edit of this artist or movie name: 'the beatles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT338 |
| ✓ | BbhRuinNames158CFWhich of the following is a humorous edit of this artist or movie name: 'raging bull'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT740 |
| × | BbhRuinNames159CFWhich of the following is a humorous edit of this artist or movie name: 'bad brains'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames159CF: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 001 OUT1 456 |
| ✓ | BbhRuinNames160CFWhich of the following is a humorous edit of this artist or movie name: 'thin lizzy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT962 |
| ✓ | BbhRuinNames161CFWhich of the following is a humorous edit of this artist or movie name: 'ben folds five'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT1 009 |
| ✓ | BbhRuinNames162CFWhich of the following is a humorous edit of this artist or movie name: 'citizen kane'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT469 |
| ✓ | BbhRuinNames163CFWhich of the following is a humorous edit of this artist or movie name: 'dave matthews band'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT334 |
| ✓ | BbhRuinNames164CFWhich of the following is a humorous edit of this artist or movie name: 'the sum of all fears'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT406 |
| ✓ | BbhRuinNames165CFWhich of the following is a humorous edit of this artist or movie name: 'city hall'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT358 |
| × | BbhRuinNames166CFWhich of the following is a humorous edit of this artist or movie name: 'the abyss'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames166CF: expected (A), got [c]
Values:
got = "c" |
Failed | IN1 001 OUT750 |
| ✓ | BbhRuinNames167CFWhich of the following is a humorous edit of this artist or movie name: 'mank'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT1 076 |
| ✓ | BbhRuinNames168CFWhich of the following is a humorous edit of this artist or movie name: 'sex pistols'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT323 |
Reasoning - BBH ruin names (humorous edits) (set 6)
38 passed3 failed |
92.7% |
IN41 275 OUT31 022 | |
| ✓ | BbhRuinNames169CGWhich of the following is a humorous edit of this artist or movie name: 'life of brian'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 865 |
| ✓ | BbhRuinNames170CGWhich of the following is a humorous edit of this artist or movie name: 'black panther'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT890 |
| ✓ | BbhRuinNames171CGWhich of the following is a humorous edit of this artist or movie name: 'goldfinger'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT766 |
| ✓ | BbhRuinNames172CGWhich of the following is a humorous edit of this artist or movie name: 'foo fighters'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 152 |
| ✓ | BbhRuinNames173CGWhich of the following is a humorous edit of this artist or movie name: 'air supply'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT309 |
| ✓ | BbhRuinNames174CGWhich of the following is a humorous edit of this artist or movie name: 'diana ross and the supremes'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 026 OUT358 |
| ✓ | BbhRuinNames175CGWhich of the following is a humorous edit of this artist or movie name: 'brokeback mountain'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT1 684 |
| ✓ | BbhRuinNames176CGWhich of the following is a humorous edit of this artist or movie name: 'godzilla vs. kong'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT370 |
| ✓ | BbhRuinNames177CGWhich of the following is a humorous edit of this artist or movie name: 'the who!'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT385 |
| ✓ | BbhRuinNames178CGWhich of the following is a humorous edit of this artist or movie name: 'the green mile'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT530 |
| ✓ | BbhRuinNames179CGWhich of the following is a humorous edit of this artist or movie name: 'the dark knight'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT388 |
| ✓ | BbhRuinNames180CGWhich of the following is a humorous edit of this artist or movie name: 'live and let die'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT203 |
| ✓ | BbhRuinNames181CGWhich of the following is a humorous edit of this artist or movie name: 'midnight cowboy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 364 |
| ✓ | BbhRuinNames182CGWhich of the following is a humorous edit of this artist or movie name: 'snow patrol'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT371 |
| ✓ | BbhRuinNames183CGWhich of the following is a humorous edit of this artist or movie name: 'the bangles'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT667 |
| ✓ | BbhRuinNames184CGWhich of the following is a humorous edit of this artist or movie name: 'state champs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT381 |
| ✓ | BbhRuinNames185CGWhich of the following is a humorous edit of this artist or movie name: 'the secret life of pets'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT1 776 |
| ✓ | BbhRuinNames186CGWhich of the following is a humorous edit of this artist or movie name: 'full metal jacket'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT538 |
| × | BbhRuinNames187CGWhich of the following is a humorous edit of this artist or movie name: 'friday the 13th'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames187CG: expected (C), got [d]
Values:
got = "d" |
Failed | IN1 023 OUT866 |
| ✓ | BbhRuinNames188CGWhich of the following is a humorous edit of this artist or movie name: 'the bay city rollers'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT317 |
| ✓ | BbhRuinNames189CGWhich of the following is a humorous edit of this artist or movie name: 'blue'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT1 342 |
| ✓ | BbhRuinNames190CGWhich of the following is a humorous edit of this artist or movie name: 'the gap band'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT493 |
| ✓ | BbhRuinNames191CGWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT471 |
| ✓ | BbhRuinNames192CGWhich of the following is a humorous edit of this artist or movie name: 'the fall of the house of usher'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT326 |
| ✓ | BbhRuinNames193CGWhich of the following is a humorous edit of this artist or movie name: 'old joy'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT561 |
| × | BbhRuinNames194CGWhich of the following is a humorous edit of this artist or movie name: 'run dmc'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames194CG: expected (D), got [c]
Values:
got = "c" |
Failed | IN1 002 OUT1 762 |
| ✓ | BbhRuinNames195CGWhich of the following is a humorous edit of this artist or movie name: 'the grapes of wrath'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT349 |
| ✓ | BbhRuinNames196CGWhich of the following is a humorous edit of this artist or movie name: 'drag me to hell'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT690 |
| ✓ | BbhRuinNames197CGWhich of the following is a humorous edit of this artist or movie name: 'talking heads'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT546 |
| ✓ | BbhRuinNames198CGWhich of the following is a humorous edit of this artist or movie name: 'paul oakenfold'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 604 |
| ✓ | BbhRuinNames199CGWhich of the following is a humorous edit of this artist or movie name: 'brief encounter'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT757 |
| ✓ | BbhRuinNames200CGWhich of the following is a humorous edit of this artist or movie name: 'a tribe called quest'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT395 |
| × | BbhRuinNames201CGWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhRuinNames201CG: expected (C), got [a]
Values:
got = "a" |
Failed | IN998 OUT1 559 |
| ✓ | BbhRuinNames202CGWhich of the following is a humorous edit of this artist or movie name: 'the wizard of oz'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT1 161 |
| ✓ | BbhRuinNames203CGWhich of the following is a humorous edit of this artist or movie name: 'run dmc'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT590 |
| ✓ | BbhRuinNames204CGWhich of the following is a humorous edit of this artist or movie name: 'the third man'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT1 010 |
| ✓ | BbhRuinNames205CGWhich of the following is a humorous edit of this artist or movie name: 'cat on a hot tin roof'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 019 OUT317 |
| ✓ | BbhRuinNames206CGWhich of the following is a humorous edit of this artist or movie name: 'notting hill'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT372 |
| ✓ | BbhRuinNames207CGWhich of the following is a humorous edit of this artist or movie name: 'good will hunting'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT348 |
| ✓ | BbhRuinNames208CGWhich of the following is a humorous edit of this artist or movie name: 'foo fighters'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT390 |
| ✓ | BbhRuinNames209CGWhich of the following is a humorous edit of this artist or movie name: 'goo goo dolls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT799 |
Reasoning - BBH ruin names (humorous edits) (set 7)
36 passed4 failed |
90.0% |
IN40 287 OUT27 513 | |
| ✓ | BbhRuinNames210CGWhich of the following is a humorous edit of this artist or movie name: 'soundgarden'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT730 |
| ✓ | BbhRuinNames211CGWhich of the following is a humorous edit of this artist or movie name: 'peter and the test tube babies'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT698 |
| ✓ | BbhRuinNames212CGWhich of the following is a humorous edit of this artist or movie name: 'blur'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT656 |
| ✓ | BbhRuinNames213CGWhich of the following is a humorous edit of this artist or movie name: 'black sabbath'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT1 197 |
| × | BbhRuinNames214CGWhich of the following is a humorous edit of this artist or movie name: 'trolls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(D)")
Context: BbhRuinNames214CG: expected (D), got [a]
Values:
got = "a" |
Failed | IN998 OUT1 152 |
| ✓ | BbhRuinNames215CGWhich of the following is a humorous edit of this artist or movie name: 'bad company'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT435 |
| ✓ | BbhRuinNames216CGWhich of the following is a humorous edit of this artist or movie name: '3:10 to yuma'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT1 046 |
| ✓ | BbhRuinNames217CGWhich of the following is a humorous edit of this artist or movie name: 'herman's hermits'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT734 |
| ✓ | BbhRuinNames218CGWhich of the following is a humorous edit of this artist or movie name: 'night of the demons'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT387 |
| ✓ | BbhRuinNames219CGWhich of the following is a humorous edit of this artist or movie name: 'the rolling stones'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT401 |
| ✓ | BbhRuinNames220CGWhich of the following is a humorous edit of this artist or movie name: 'blur'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT378 |
| ✓ | BbhRuinNames221CGWhich of the following is a humorous edit of this artist or movie name: 'a matter of life and death'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT296 |
| ✓ | BbhRuinNames222CGWhich of the following is a humorous edit of this artist or movie name: 'singin' in the rain'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT322 |
| ✓ | BbhRuinNames223CGWhich of the following is a humorous edit of this artist or movie name: 'the man with two brains'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT421 |
| ✓ | BbhRuinNames224CGWhich of the following is a humorous edit of this artist or movie name: 'born free'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT407 |
| ✓ | BbhRuinNames225CGWhich of the following is a humorous edit of this artist or movie name: 'the moody blues'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT646 |
| ✓ | BbhRuinNames226CGWhich of the following is a humorous edit of this artist or movie name: 'jojo rabbit'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT885 |
| ✓ | BbhRuinNames227CGWhich of the following is a humorous edit of this artist or movie name: 'reservoir dogs'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT522 |
| ✓ | BbhRuinNames228CGWhich of the following is a humorous edit of this artist or movie name: 'thunder road'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT354 |
| ✓ | BbhRuinNames229CGWhich of the following is a humorous edit of this artist or movie name: 'star wars'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT783 |
| ✓ | BbhRuinNames230CGWhich of the following is a humorous edit of this artist or movie name: 'deadpool'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT412 |
| ✓ | BbhRuinNames231CGWhich of the following is a humorous edit of this artist or movie name: 'the lady vanishes'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT436 |
| ✓ | BbhRuinNames232CGWhich of the following is a humorous edit of this artist or movie name: 'steely dan'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT1 153 |
| ✓ | BbhRuinNames233CGWhich of the following is a humorous edit of this artist or movie name: 'the men who stare at goats'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT580 |
| ✓ | BbhRuinNames234CGWhich of the following is a humorous edit of this artist or movie name: 'the stone roses'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT1 228 |
| ✓ | BbhRuinNames235CGWhich of the following is a humorous edit of this artist or movie name: 'nine inch nails'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT751 |
| ✓ | BbhRuinNames236CGWhich of the following is a humorous edit of this artist or movie name: 'billy idol'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT360 |
| ✓ | BbhRuinNames237CGWhich of the following is a humorous edit of this artist or movie name: 'schindler's list'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT453 |
| ✓ | BbhRuinNames238CGWhich of the following is a humorous edit of this artist or movie name: 'sex pistols'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT790 |
| ✓ | BbhRuinNames239CGWhich of the following is a humorous edit of this artist or movie name: 'gone with the wind'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT344 |
| × | BbhRuinNames240CGWhich of the following is a humorous edit of this artist or movie name: 'it's a wonderful life'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames240CG: expected (A), got [b]
Values:
got = "b" |
Failed | IN1 016 OUT1 283 |
| × | BbhRuinNames241CGWhich of the following is a humorous edit of this artist or movie name: 'guns n' roses'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhRuinNames241CG: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 011 OUT1 811 |
| ✓ | BbhRuinNames242CGWhich of the following is a humorous edit of this artist or movie name: 'witness'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 070 |
| × | BbhRuinNames243CGWhich of the following is a humorous edit of this artist or movie name: 'killing joke'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhRuinNames243CG: expected (A), got [c]
Values:
got = "c" |
Failed | IN1 000 OUT559 |
| ✓ | BbhRuinNames244CGWhich of the following is a humorous edit of this artist or movie name: 'panic! at the disco'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 015 OUT432 |
| ✓ | BbhRuinNames245CGWhich of the following is a humorous edit of this artist or movie name: 'spice girls'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT937 |
| ✓ | BbhRuinNames246CGWhich of the following is a humorous edit of this artist or movie name: 'the police'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT354 |
| ✓ | BbhRuinNames247CGWhich of the following is a humorous edit of this artist or movie name: 'spandau ballet'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT420 |
| ✓ | BbhRuinNames248CGWhich of the following is a humorous edit of this artist or movie name: 'toto'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT1 385 |
| ✓ | BbhRuinNames249CGWhich of the following is a humorous edit of this artist or movie name: 'system of a down'?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT305 |
Reasoning - BBH snarks - detect sarcasm (set 1)
26 passed6 failed |
81.3% |
IN33 458 OUT18 673 | |
| × | BbhSalientTranslation243CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSalientTranslation243CI: expected (A), got [e]
Values:
got = "e" |
Failed | IN1 204 OUT1 484 |
| ✓ | BbhSalientTranslation244CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 180 OUT835 |
| × | BbhSalientTranslation245CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSalientTranslation245CI: expected (B), got [d]
Values:
got = "d" |
Failed | IN1 194 OUT954 |
| ✓ | BbhSalientTranslation246CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 237 OUT594 |
| ✓ | BbhSalientTranslation247CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 192 OUT495 |
| × | BbhSalientTranslation248CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(E)")
Context: BbhSalientTranslation248CI: expected (E), got [a]
Values:
got = "a" |
Failed | IN1 187 OUT1 127 |
| × | BbhSalientTranslation249CIidentify the type of error in a translation1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(C)")
Context: BbhSalientTranslation249CI: expected (C), got [a]
Values:
got = "a" |
Failed | IN1 234 OUT1 069 |
| ✓ | BbhSnarks10CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT293 |
| ✓ | BbhSnarks11CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT456 |
| ✓ | BbhSnarks12CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT396 |
| ✓ | BbhSnarks13CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT364 |
| ✓ | BbhSnarks14CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT319 |
| ✓ | BbhSnarks15CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT278 |
| ✓ | BbhSnarks16CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT276 |
| ✓ | BbhSnarks17CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT368 |
| ✓ | BbhSnarks18CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT269 |
| ✓ | BbhSnarks19CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT1 008 |
| × | BbhSnarks20CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSnarks20CI: expected (A), got [b]
Values:
got = "b" |
Failed | IN987 OUT784 |
| ✓ | BbhSnarks21CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT361 |
| × | BbhSnarks22CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks22CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN996 OUT1 437 |
| ✓ | BbhSnarks23CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT684 |
| ✓ | BbhSnarks24CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT348 |
| ✓ | BbhSnarks25CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT1 263 |
| ✓ | BbhSnarks26CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT530 |
| ✓ | BbhSnarks27CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT304 |
| ✓ | BbhSnarks28CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT385 |
| ✓ | BbhSnarks29CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT501 |
| ✓ | BbhSnarks30CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT221 |
| ✓ | BbhSnarks31CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT275 |
| ✓ | BbhSnarks32CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT399 |
| ✓ | BbhSnarks8CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT313 |
| ✓ | BbhSnarks9CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT283 |
Reasoning - BBH snarks - detect sarcasm (set 2)
28 passed4 failed |
87.5% |
IN32 212 OUT16 792 | |
| ✓ | BbhSnarks33CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT319 |
| ✓ | BbhSnarks34CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT287 |
| ✓ | BbhSnarks35CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT312 |
| ✓ | BbhSnarks36CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT311 |
| ✓ | BbhSnarks37CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT334 |
| ✓ | BbhSnarks38CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT379 |
| ✓ | BbhSnarks39CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT308 |
| ✓ | BbhSnarks40CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT1 229 |
| ✓ | BbhSnarks41CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT644 |
| ✓ | BbhSnarks42CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT357 |
| × | BbhSnarks43CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks43CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 006 OUT855 |
| ✓ | BbhSnarks44CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN987 OUT979 |
| ✓ | BbhSnarks45CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT528 |
| ✓ | BbhSnarks46CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 050 OUT300 |
| ✓ | BbhSnarks47CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT546 |
| ✓ | BbhSnarks48CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN997 OUT876 |
| × | BbhSnarks49CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSnarks49CI: expected (A), got [b]
Values:
got = "b" |
Failed | IN990 OUT839 |
| ✓ | BbhSnarks50CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT1 591 |
| ✓ | BbhSnarks51CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT536 |
| ✓ | BbhSnarks52CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT422 |
| ✓ | BbhSnarks53CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT448 |
| ✓ | BbhSnarks54CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT223 |
| ✓ | BbhSnarks55CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT346 |
| ✓ | BbhSnarks56CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT330 |
| ✓ | BbhSnarks57CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT209 |
| × | BbhSnarks58CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks58CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN998 OUT311 |
| ✓ | BbhSnarks59CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT244 |
| × | BbhSnarks60CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks60CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN998 OUT1 446 |
| ✓ | BbhSnarks61CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 034 OUT259 |
| ✓ | BbhSnarks62CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT385 |
| ✓ | BbhSnarks63CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT322 |
| ✓ | BbhSnarks64CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT317 |
Reasoning - BBH snarks - detect sarcasm (set 3)
27 passed3 failed2 skipped |
90.0% |
IN30 151 OUT18 992 | |
| ✓ | BbhSnarks65CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN978 OUT281 |
| × | BbhSnarks66CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks66CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 008 OUT948 |
| ✓ | BbhSnarks67CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT340 |
| ✓ | BbhSnarks68CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT281 |
| ✓ | BbhSnarks69CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 017 OUT952 |
| ✓ | BbhSnarks70CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT251 |
| ✓ | BbhSnarks71CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT587 |
| ✓ | BbhSnarks72CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 001 OUT1 008 |
| ✓ | BbhSnarks73CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT266 |
| ✓ | BbhSnarks74CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT1 474 |
| ✓ | BbhSnarks75CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT345 |
| ✓ | BbhSnarks76CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT348 |
| ✓ | BbhSnarks77CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 042 OUT1 113 |
| ✓ | BbhSnarks78CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 016 OUT1 284 |
| ✓ | BbhSnarks79CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT768 |
| ✓ | BbhSnarks80CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT334 |
| × | BbhSnarks81CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSnarks81CI: expected (A), got [b]
Values:
got = "b" |
Failed | IN990 OUT612 |
| ✓ | BbhSnarks82CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT1 349 |
| ✓ | BbhSnarks83CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT359 |
| ✓ | BbhSnarks84CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT428 |
| ✓ | BbhSnarks85CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT519 |
| – | BbhSnarks86CIWhich statement is sarcastic?0 AI queries
0 tool callscategory:Reasoninglevel:Hardsource:BBH Skipped: non-deterministic answer: both options are identical |
Skipped | IN0 OUT0 |
| ✓ | BbhSnarks87CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT347 |
| – | BbhSnarks88CIWhich statement is sarcastic?0 AI queries
0 tool callscategory:Reasoninglevel:Hardsource:BBH Skipped: non-deterministic answer: malformed item: single truncated option |
Skipped | IN0 OUT0 |
| ✓ | BbhSnarks89CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT384 |
| ✓ | BbhSnarks90CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT374 |
| ✓ | BbhSnarks91CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT342 |
| ✓ | BbhSnarks92CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT307 |
| × | BbhSnarks93CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSnarks93CI: expected (A), got [b]
Values:
got = "b" |
Failed | IN997 OUT2 292 |
| ✓ | BbhSnarks94CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT453 |
| ✓ | BbhSnarks95CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT266 |
| ✓ | BbhSnarks96CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT380 |
Reasoning - BBH snarks - detect sarcasm (set 4)
28 passed4 failed |
87.5% |
IN31 981 OUT16 633 | |
| ✓ | BbhSnarks100CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT303 |
| ✓ | BbhSnarks101CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT344 |
| ✓ | BbhSnarks102CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT269 |
| ✓ | BbhSnarks103CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT400 |
| ✓ | BbhSnarks104CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN995 OUT315 |
| ✓ | BbhSnarks105CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT346 |
| ✓ | BbhSnarks106CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 009 OUT1 119 |
| ✓ | BbhSnarks107CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN983 OUT345 |
| ✓ | BbhSnarks108CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 022 OUT334 |
| ✓ | BbhSnarks109CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 025 OUT340 |
| ✓ | BbhSnarks110CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT336 |
| ✓ | BbhSnarks111CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT1 439 |
| ✓ | BbhSnarks112CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT452 |
| ✓ | BbhSnarks113CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 003 OUT490 |
| ✓ | BbhSnarks114CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN979 OUT231 |
| ✓ | BbhSnarks115CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT297 |
| ✓ | BbhSnarks116CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT254 |
| × | BbhSnarks117CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks117CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 001 OUT473 |
| ✓ | BbhSnarks118CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT327 |
| ✓ | BbhSnarks119CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT319 |
| ✓ | BbhSnarks120CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT261 |
| ✓ | BbhSnarks121CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT333 |
| × | BbhSnarks122CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks122CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN992 OUT977 |
| ✓ | BbhSnarks123CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT254 |
| ✓ | BbhSnarks124CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT749 |
| × | BbhSnarks125CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks125CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 002 OUT377 |
| ✓ | BbhSnarks126CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN980 OUT243 |
| ✓ | BbhSnarks127CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 011 OUT299 |
| × | BbhSnarks128CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(A)")
Context: BbhSnarks128CI: expected (A), got [b]
Values:
got = "b" |
Failed | IN1 000 OUT936 |
| ✓ | BbhSnarks97CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN985 OUT850 |
| ✓ | BbhSnarks98CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT357 |
| ✓ | BbhSnarks99CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT2 264 |
Reasoning - BBH snarks - detect sarcasm (set 5)
29 passed1 failed |
96.7% |
IN30 115 OUT14 190 | |
| ✓ | BbhSnarks129CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT717 |
| ✓ | BbhSnarks130CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT350 |
| ✓ | BbhSnarks131CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT901 |
| ✓ | BbhSnarks132CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT348 |
| ✓ | BbhSnarks133CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 037 OUT431 |
| ✓ | BbhSnarks134CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 030 OUT341 |
| ✓ | BbhSnarks135CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 005 OUT427 |
| ✓ | BbhSnarks136CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN986 OUT268 |
| ✓ | BbhSnarks137CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 031 OUT309 |
| ✓ | BbhSnarks138CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT360 |
| ✓ | BbhSnarks139CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 033 OUT306 |
| ✓ | BbhSnarks140CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT298 |
| ✓ | BbhSnarks141CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT301 |
| ✓ | BbhSnarks142CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT284 |
| ✓ | BbhSnarks143CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT297 |
| × | BbhSnarks144CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks144CI: expected (B), got [a]
Values:
got = "a" |
Failed | IN1 000 OUT1 271 |
| ✓ | BbhSnarks145CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT573 |
| ✓ | BbhSnarks146CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT376 |
| ✓ | BbhSnarks147CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN999 OUT844 |
| ✓ | BbhSnarks148CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT381 |
| ✓ | BbhSnarks149CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT246 |
| ✓ | BbhSnarks150CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT403 |
| ✓ | BbhSnarks151CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN984 OUT314 |
| ✓ | BbhSnarks152CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 018 OUT1 264 |
| ✓ | BbhSnarks153CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT434 |
| ✓ | BbhSnarks154CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN991 OUT299 |
| ✓ | BbhSnarks155CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT463 |
| ✓ | BbhSnarks156CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 024 OUT382 |
| ✓ | BbhSnarks157CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT499 |
| ✓ | BbhSnarks158CIWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 007 OUT503 |
Reasoning - BBH snarks - detect sarcasm (set 6)
28 passed4 failed |
87.5% |
IN31 659 OUT15 828 | |
| ✓ | BbhSnarks159CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT288 |
| ✓ | BbhSnarks160CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT664 |
| ✓ | BbhSnarks161CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN993 OUT252 |
| ✓ | BbhSnarks162CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT336 |
| ✓ | BbhSnarks163CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT235 |
| ✓ | BbhSnarks164CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT648 |
| ✓ | BbhSnarks165CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN982 OUT208 |
| ✓ | BbhSnarks166CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN988 OUT306 |
| ✓ | BbhSnarks167CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT323 |
| × | BbhSnarks168CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: got == Norm("(B)")
Context: BbhSnarks168CJ expected (B), got [a]
Values:
got = "a" |
Failed | IN992 OUT1 546 |
| ✓ | BbhSnarks169CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 013 OUT581 |
| ✓ | BbhSnarks170CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT505 |
| ✓ | BbhSnarks171CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT362 |
| ✓ | BbhSnarks172CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN990 OUT444 |
| ✓ | BbhSnarks173CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 020 OUT382 |
| ✓ | BbhSnarks174CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT448 |
| ✓ | BbhSnarks175CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 012 OUT321 |
| ✓ | BbhSnarks176CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 014 OUT320 |
| ✓ | BbhSnarks177CJWhich statement is sarcastic?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT323 |
| ✓ | BbhSportsUnderstanding10CJIs the following sentence plausible? "Carson Wentz took to the ice."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN971 OUT245 |
| ✓ | BbhSportsUnderstanding11CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN974 OUT1 727 |
| ✓ | BbhSportsUnderstanding12CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN977 OUT496 |
| ✓ | BbhSportsUnderstanding13CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN972 OUT948 |
| ✓ | BbhSportsUnderstanding14CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN972 OUT429 |
| ✓ | BbhSportsUnderstanding15CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN975 OUT477 |
| ✓ | BbhSportsUnderstanding16CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN971 OUT147 |
| ✓ | BbhSportsUnderstanding17CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN970 OUT766 |
| ✓ | BbhSportsUnderstanding18CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN969 OUT422 |
| × | BbhSportsUnderstanding19CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding19CJ expected yes, got [no]
Values:
got = "no" |
Failed | IN976 OUT249 |
| ✓ | BbhSportsUnderstanding20CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN970 OUT296 |
| × | BbhSportsUnderstanding8CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding8CJ expected yes, got [no]
Values:
got = "no" |
Failed | IN975 OUT831 |
| × | BbhSportsUnderstanding9CJjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("no")
Context: BbhSportsUnderstanding9CJ expected no, got [yes]
Values:
got = "yes" |
Failed | IN971 OUT303 |
Reasoning - BBH temporal sequences (set 1)
28 passed4 failed |
87.5% |
IN35 191 OUT15 503 | |
| ✓ | BbhSportsUnderstanding243CKIs the following sentence plausible? "Blake Snell hit a single."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN969 OUT450 |
| × | BbhSportsUnderstanding244CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding244CK expected yes, got [no]
Values:
got = "no" |
Failed | IN976 OUT454 |
| ✓ | BbhSportsUnderstanding245CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN974 OUT276 |
| × | BbhSportsUnderstanding246CKIs the following sentence plausible? "Pepe converted the first down."1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("no")
Context: BbhSportsUnderstanding246CK expected no, got [yes]
Values:
got = "yes" |
Failed | IN969 OUT422 |
| × | BbhSportsUnderstanding247CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding247CK expected yes, got [false]
Values:
got = "false" |
Failed | IN977 OUT897 |
| ✓ | BbhSportsUnderstanding248CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH |
Passed | IN971 OUT536 |
| × | BbhSportsUnderstanding249CKjudge the plausibility of a sports sentence1 AI query
0 tool callscategory:Reasoninglevel:Mediumsource:BBH Assertion expression evaluated to false: got == Norm("yes")
Context: BbhSportsUnderstanding249CK expected yes, got [true]
Values:
got = "true" |
Failed | IN972 OUT294 |
| ✓ | BbhTemporalSequences10CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT434 |
| ✓ | BbhTemporalSequences11CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT487 |
| ✓ | BbhTemporalSequences12CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT707 |
| ✓ | BbhTemporalSequences13CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 140 OUT447 |
| ✓ | BbhTemporalSequences14CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT475 |
| ✓ | BbhTemporalSequences15CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT498 |
| ✓ | BbhTemporalSequences16CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT437 |
| ✓ | BbhTemporalSequences17CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT437 |
| ✓ | BbhTemporalSequences18CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT497 |
| ✓ | BbhTemporalSequences19CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT434 |
| ✓ | BbhTemporalSequences20CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT549 |
| ✓ | BbhTemporalSequences21CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT596 |
| ✓ | BbhTemporalSequences22CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT551 |
| ✓ | BbhTemporalSequences23CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT560 |
| ✓ | BbhTemporalSequences24CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT455 |
| ✓ | BbhTemporalSequences25CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT527 |
| ✓ | BbhTemporalSequences26CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT478 |
| ✓ | BbhTemporalSequences27CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT367 |
| ✓ | BbhTemporalSequences28CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT575 |
| ✓ | BbhTemporalSequences29CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT494 |
| ✓ | BbhTemporalSequences30CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT440 |
| ✓ | BbhTemporalSequences31CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT485 |
| ✓ | BbhTemporalSequences32CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT447 |
| ✓ | BbhTemporalSequences8CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 159 OUT424 |
| ✓ | BbhTemporalSequences9CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT373 |
Reasoning - BBH temporal sequences (set 2)
30 passed |
100.0% |
IN33 963 OUT14 571 | |
| ✓ | BbhTemporalSequences33CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT355 |
| ✓ | BbhTemporalSequences34CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT496 |
| ✓ | BbhTemporalSequences35CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT366 |
| ✓ | BbhTemporalSequences36CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT465 |
| ✓ | BbhTemporalSequences37CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT430 |
| ✓ | BbhTemporalSequences38CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT572 |
| ✓ | BbhTemporalSequences39CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT453 |
| ✓ | BbhTemporalSequences40CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT590 |
| ✓ | BbhTemporalSequences41CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 106 OUT411 |
| ✓ | BbhTemporalSequences42CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT509 |
| ✓ | BbhTemporalSequences43CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT448 |
| ✓ | BbhTemporalSequences44CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT473 |
| ✓ | BbhTemporalSequences45CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT467 |
| ✓ | BbhTemporalSequences46CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT511 |
| ✓ | BbhTemporalSequences47CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT371 |
| ✓ | BbhTemporalSequences48CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT632 |
| ✓ | BbhTemporalSequences49CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT573 |
| ✓ | BbhTemporalSequences50CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT458 |
| ✓ | BbhTemporalSequences51CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT403 |
| ✓ | BbhTemporalSequences52CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT369 |
| ✓ | BbhTemporalSequences53CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT469 |
| ✓ | BbhTemporalSequences54CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT726 |
| ✓ | BbhTemporalSequences55CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT494 |
| ✓ | BbhTemporalSequences56CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 158 OUT492 |
| ✓ | BbhTemporalSequences57CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT372 |
| ✓ | BbhTemporalSequences58CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT518 |
| ✓ | BbhTemporalSequences59CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT463 |
| ✓ | BbhTemporalSequences60CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT627 |
| ✓ | BbhTemporalSequences61CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT507 |
| ✓ | BbhTemporalSequences62CKfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT551 |
Reasoning - BBH temporal sequences (set 3)
32 passed |
100.0% |
IN36 175 OUT15 185 | |
| ✓ | BbhTemporalSequences63CLfind the time window when Samantha could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT475 |
| ✓ | BbhTemporalSequences64CLfind the time window when Betty could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT571 |
| ✓ | BbhTemporalSequences65CLfind the time window when William could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT565 |
| ✓ | BbhTemporalSequences66CLfind the time window when James could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT429 |
| ✓ | BbhTemporalSequences67CLfind the time window when Samantha could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT561 |
| ✓ | BbhTemporalSequences68CLfind the time window when William could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT395 |
| ✓ | BbhTemporalSequences69CLfind the time window when Kimberly could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT353 |
| ✓ | BbhTemporalSequences70CLfind the time window when Susan could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT550 |
| ✓ | BbhTemporalSequences71CLfind the time window when James could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT449 |
| ✓ | BbhTemporalSequences72CLfind the time window when Ashley could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT387 |
| ✓ | BbhTemporalSequences73CLfind the time window when Jennifer could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT488 |
| ✓ | BbhTemporalSequences74CLfind the time window when James could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT561 |
| ✓ | BbhTemporalSequences75CLfind the time window when Anthony could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT423 |
| ✓ | BbhTemporalSequences76CLfind the time window when Thomas could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 159 OUT504 |
| ✓ | BbhTemporalSequences77CLfind the time window when Jennifer could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT365 |
| ✓ | BbhTemporalSequences78CLfind the time window when Sarah could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT399 |
| ✓ | BbhTemporalSequences79CLfind the time window when Susan could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT380 |
| ✓ | BbhTemporalSequences80CLfind the time window when William could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT478 |
| ✓ | BbhTemporalSequences81CLfind the time window when Jessica could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT507 |
| ✓ | BbhTemporalSequences82CLfind the time window when Kimberly could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT412 |
| ✓ | BbhTemporalSequences83CLfind the time window when Elizabeth could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT413 |
| ✓ | BbhTemporalSequences84CLfind the time window when James could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT584 |
| ✓ | BbhTemporalSequences85CLfind the time window when Michael could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT466 |
| ✓ | BbhTemporalSequences86CLfind the time window when Sarah could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 158 OUT502 |
| ✓ | BbhTemporalSequences87CLfind the time window when Mark could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 159 OUT567 |
| ✓ | BbhTemporalSequences88CLfind the time window when Sean could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT579 |
| ✓ | BbhTemporalSequences89CLfind the time window when James could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT538 |
| ✓ | BbhTemporalSequences90CLfind the time window when Jessica could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT476 |
| ✓ | BbhTemporalSequences91CLfind the time window when Tiffany could have gone to the library1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT490 |
| ✓ | BbhTemporalSequences92CLfind the time window when James could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT411 |
| ✓ | BbhTemporalSequences93CLfind the time window when Nancy could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT286 |
| ✓ | BbhTemporalSequences94CLfind the time window when Tiffany could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT621 |
Reasoning - BBH temporal sequences (set 4)
32 passed |
100.0% |
IN36 069 OUT16 010 | |
| ✓ | BbhTemporalSequences100CLfind the time window when Steven could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT465 |
| ✓ | BbhTemporalSequences101CLfind the time window when Jessica could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT678 |
| ✓ | BbhTemporalSequences102CLfind the time window when Jessica could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT639 |
| ✓ | BbhTemporalSequences103CLfind the time window when Emily could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT449 |
| ✓ | BbhTemporalSequences104CLfind the time window when Anthony could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT371 |
| ✓ | BbhTemporalSequences105CLfind the time window when Mark could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT381 |
| ✓ | BbhTemporalSequences106CLfind the time window when William could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT662 |
| ✓ | BbhTemporalSequences107CLfind the time window when Hannah could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT470 |
| ✓ | BbhTemporalSequences108CLfind the time window when William could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT530 |
| ✓ | BbhTemporalSequences109CLfind the time window when Nancy could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT687 |
| ✓ | BbhTemporalSequences110CLfind the time window when Andrew could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT482 |
| ✓ | BbhTemporalSequences111CLfind the time window when David could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT508 |
| ✓ | BbhTemporalSequences112CLfind the time window when Anthony could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 104 OUT368 |
| ✓ | BbhTemporalSequences113CLfind the time window when Sarah could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT461 |
| ✓ | BbhTemporalSequences114CLfind the time window when Mary could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT430 |
| ✓ | BbhTemporalSequences115CLfind the time window when Mark could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT410 |
| ✓ | BbhTemporalSequences116CLfind the time window when William could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT475 |
| ✓ | BbhTemporalSequences117CLfind the time window when Steven could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT434 |
| ✓ | BbhTemporalSequences118CLfind the time window when Leslie could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT681 |
| ✓ | BbhTemporalSequences119CLfind the time window when Sarah could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT411 |
| ✓ | BbhTemporalSequences120CLfind the time window when Andrew could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT379 |
| ✓ | BbhTemporalSequences121CLfind the time window when David could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT487 |
| ✓ | BbhTemporalSequences122CLfind the time window when Jessica could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 158 OUT441 |
| ✓ | BbhTemporalSequences123CLfind the time window when David could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT569 |
| ✓ | BbhTemporalSequences124CLfind the time window when Jessica could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT475 |
| ✓ | BbhTemporalSequences125CLfind the time window when Mary could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT540 |
| ✓ | BbhTemporalSequences126CLfind the time window when Michael could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT557 |
| ✓ | BbhTemporalSequences95CLfind the time window when Jennifer could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT486 |
| ✓ | BbhTemporalSequences96CLfind the time window when Sean could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT463 |
| ✓ | BbhTemporalSequences97CLfind the time window when Samantha could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT713 |
| ✓ | BbhTemporalSequences98CLfind the time window when Ashley could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT405 |
| ✓ | BbhTemporalSequences99CLfind the time window when Sarah could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 139 OUT503 |
Reasoning - BBH temporal sequences (set 5)
32 passed |
100.0% |
IN36 213 OUT15 269 | |
| ✓ | BbhTemporalSequences127CLfind the time window when David could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT326 |
| ✓ | BbhTemporalSequences128CLfind the time window when Andrew could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT429 |
| ✓ | BbhTemporalSequences129CLfind the time window when Leslie could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT500 |
| ✓ | BbhTemporalSequences130CLfind the time window when Mary could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT485 |
| ✓ | BbhTemporalSequences131CLfind the time window when Ashley could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 145 OUT567 |
| ✓ | BbhTemporalSequences132CLfind the time window when Samantha could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT433 |
| ✓ | BbhTemporalSequences133CLfind the time window when Elizabeth could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT508 |
| ✓ | BbhTemporalSequences134CLfind the time window when Sean could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT436 |
| ✓ | BbhTemporalSequences135CLfind the time window when Elizabeth could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT395 |
| ✓ | BbhTemporalSequences136CLfind the time window when Michael could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT475 |
| ✓ | BbhTemporalSequences137CLfind the time window when Andrew could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT475 |
| ✓ | BbhTemporalSequences138CLfind the time window when Nancy could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT528 |
| ✓ | BbhTemporalSequences139CLfind the time window when Jason could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT580 |
| ✓ | BbhTemporalSequences140CLfind the time window when Anthony could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT519 |
| ✓ | BbhTemporalSequences141CLfind the time window when Leslie could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT452 |
| ✓ | BbhTemporalSequences142CLfind the time window when David could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT758 |
| ✓ | BbhTemporalSequences143CLfind the time window when Hannah could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT435 |
| ✓ | BbhTemporalSequences144CLfind the time window when Sarah could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT411 |
| ✓ | BbhTemporalSequences145CLfind the time window when Andrew could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT442 |
| ✓ | BbhTemporalSequences146CLfind the time window when William could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT461 |
| ✓ | BbhTemporalSequences147CLfind the time window when Elizabeth could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT419 |
| ✓ | BbhTemporalSequences148CLfind the time window when Sarah could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT574 |
| ✓ | BbhTemporalSequences149CLfind the time window when David could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT385 |
| ✓ | BbhTemporalSequences150CLfind the time window when Jason could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT333 |
| ✓ | BbhTemporalSequences151CLfind the time window when Anthony could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT576 |
| ✓ | BbhTemporalSequences152CLfind the time window when Sarah could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 138 OUT421 |
| ✓ | BbhTemporalSequences153CLfind the time window when Hannah could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT496 |
| ✓ | BbhTemporalSequences154CLfind the time window when Mark could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT471 |
| ✓ | BbhTemporalSequences155CLfind the time window when Sarah could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT437 |
| ✓ | BbhTemporalSequences156CLfind the time window when James could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT556 |
| ✓ | BbhTemporalSequences157CLfind the time window when Emily could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT487 |
| ✓ | BbhTemporalSequences158CLfind the time window when David could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT499 |
Reasoning - BBH temporal sequences (set 6)
32 passed |
100.0% |
IN36 181 OUT14 956 | |
| ✓ | BbhTemporalSequences159CLfind the time window when Jennifer could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT429 |
| ✓ | BbhTemporalSequences160CLfind the time window when Ashley could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 144 OUT472 |
| ✓ | BbhTemporalSequences161CLfind the time window when William could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 125 OUT413 |
| ✓ | BbhTemporalSequences162CLfind the time window when Jennifer could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT490 |
| ✓ | BbhTemporalSequences163CLfind the time window when Jennifer could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT473 |
| ✓ | BbhTemporalSequences164CLfind the time window when Jennifer could have gone to the orchestra hall1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT379 |
| ✓ | BbhTemporalSequences165CLfind the time window when Nancy could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT453 |
| ✓ | BbhTemporalSequences166CLfind the time window when Samantha could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT623 |
| ✓ | BbhTemporalSequences167CLfind the time window when Tiffany could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT435 |
| ✓ | BbhTemporalSequences168CLfind the time window when Leslie could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 157 OUT434 |
| ✓ | BbhTemporalSequences169CLfind the time window when Hannah could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 150 OUT462 |
| ✓ | BbhTemporalSequences170CLfind the time window when Anthony could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT535 |
| ✓ | BbhTemporalSequences171CLfind the time window when Hannah could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT530 |
| ✓ | BbhTemporalSequences172CLfind the time window when Leslie could have gone to the swimming pool1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT533 |
| ✓ | BbhTemporalSequences173CLfind the time window when Anthony could have gone to the bookstore1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT326 |
| ✓ | BbhTemporalSequences174CLfind the time window when Thomas could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT581 |
| ✓ | BbhTemporalSequences175CLfind the time window when Jason could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT602 |
| ✓ | BbhTemporalSequences176CLfind the time window when Betty could have gone to the bakery1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT425 |
| ✓ | BbhTemporalSequences177CLfind the time window when Ashley could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT423 |
| ✓ | BbhTemporalSequences178CLfind the time window when David could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 138 OUT429 |
| ✓ | BbhTemporalSequences179CLfind the time window when Leslie could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT411 |
| ✓ | BbhTemporalSequences180CLfind the time window when Ashley could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT488 |
| ✓ | BbhTemporalSequences181CLfind the time window when Michael could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT459 |
| ✓ | BbhTemporalSequences182CLfind the time window when Jason could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT423 |
| ✓ | BbhTemporalSequences183CLfind the time window when Sean could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 127 OUT374 |
| ✓ | BbhTemporalSequences184CLfind the time window when Sarah could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT702 |
| ✓ | BbhTemporalSequences185CLfind the time window when Andrew could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT528 |
| ✓ | BbhTemporalSequences186CLfind the time window when James could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 129 OUT422 |
| ✓ | BbhTemporalSequences187CLfind the time window when Ashley could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT403 |
| ✓ | BbhTemporalSequences188CLfind the time window when Ashley could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 151 OUT510 |
| ✓ | BbhTemporalSequences189CLfind the time window when Sarah could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 136 OUT394 |
| ✓ | BbhTemporalSequences190CLfind the time window when Michael could have gone to the football field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT395 |
Reasoning - BBH temporal sequences (set 7)
30 passed |
100.0% |
IN33 866 OUT13 573 | |
| ✓ | BbhTemporalSequences191CLfind the time window when Anthony could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT394 |
| ✓ | BbhTemporalSequences192CLfind the time window when William could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT493 |
| ✓ | BbhTemporalSequences193CLfind the time window when Susan could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 147 OUT437 |
| ✓ | BbhTemporalSequences194CLfind the time window when Sarah could have gone to the dance studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT438 |
| ✓ | BbhTemporalSequences195CLfind the time window when Emily could have gone to the park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT535 |
| ✓ | BbhTemporalSequences196CLfind the time window when Emily could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT366 |
| ✓ | BbhTemporalSequences197CLfind the time window when Sean could have gone to the art show1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 133 OUT436 |
| ✓ | BbhTemporalSequences198CLfind the time window when Elizabeth could have gone to the movies1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 143 OUT371 |
| ✓ | BbhTemporalSequences199CLfind the time window when William could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT464 |
| ✓ | BbhTemporalSequences200CLfind the time window when Thomas could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT435 |
| ✓ | BbhTemporalSequences201CLfind the time window when Susan could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 121 OUT463 |
| ✓ | BbhTemporalSequences202CLfind the time window when Kimberly could have gone to the construction site1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 160 OUT443 |
| ✓ | BbhTemporalSequences203CLfind the time window when Leslie could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT527 |
| ✓ | BbhTemporalSequences204CLfind the time window when William could have gone to the amusement park1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT474 |
| ✓ | BbhTemporalSequences205CLfind the time window when Andrew could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT488 |
| ✓ | BbhTemporalSequences206CLfind the time window when Jennifer could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT385 |
| ✓ | BbhTemporalSequences207CLfind the time window when Susan could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT421 |
| ✓ | BbhTemporalSequences208CLfind the time window when James could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT564 |
| ✓ | BbhTemporalSequences209CLfind the time window when Tiffany could have gone to the clothing store1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT416 |
| ✓ | BbhTemporalSequences210CLfind the time window when Andrew could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT463 |
| ✓ | BbhTemporalSequences211CLfind the time window when Thomas could have gone to the restaurant1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT456 |
| ✓ | BbhTemporalSequences212CLfind the time window when David could have gone to the museum1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 152 OUT458 |
| ✓ | BbhTemporalSequences213CLfind the time window when Elizabeth could have gone to the art studio1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT385 |
| ✓ | BbhTemporalSequences214CLfind the time window when Jason could have gone to the physics classroom1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT429 |
| ✓ | BbhTemporalSequences215CLfind the time window when Sarah could have gone to the gas station1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT490 |
| ✓ | BbhTemporalSequences216CLfind the time window when William could have gone to the coffee shop1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT481 |
| ✓ | BbhTemporalSequences217CLfind the time window when Ashley could have gone to the basketball court1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT387 |
| ✓ | BbhTemporalSequences218CLfind the time window when William could have gone to the market1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT445 |
| ✓ | BbhTemporalSequences219CLfind the time window when Sarah could have gone to the soccer field1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT597 |
| ✓ | BbhTemporalSequences220CLfind the time window when Nancy could have gone to the beach1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT432 |
Reasoning - BBH temporal sequences (set 8)
29 passed |
100.0% |
IN33 017 OUT13 750 | |
| ✓ | BbhTemporalSequences221CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT489 |
| ✓ | BbhTemporalSequences222CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT303 |
| ✓ | BbhTemporalSequences223CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT478 |
| ✓ | BbhTemporalSequences224CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 135 OUT492 |
| ✓ | BbhTemporalSequences225CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT542 |
| ✓ | BbhTemporalSequences226CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT569 |
| ✓ | BbhTemporalSequences227CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 137 OUT558 |
| ✓ | BbhTemporalSequences228CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 153 OUT496 |
| ✓ | BbhTemporalSequences229CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 123 OUT508 |
| ✓ | BbhTemporalSequences230CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 149 OUT364 |
| ✓ | BbhTemporalSequences231CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT427 |
| ✓ | BbhTemporalSequences232CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT397 |
| ✓ | BbhTemporalSequences233CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT495 |
| ✓ | BbhTemporalSequences234CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT456 |
| ✓ | BbhTemporalSequences235CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 115 OUT459 |
| ✓ | BbhTemporalSequences236CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 113 OUT622 |
| ✓ | BbhTemporalSequences237CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 148 OUT383 |
| ✓ | BbhTemporalSequences238CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT606 |
| ✓ | BbhTemporalSequences239CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 131 OUT342 |
| ✓ | BbhTemporalSequences240CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 155 OUT474 |
| ✓ | BbhTemporalSequences241CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 156 OUT474 |
| ✓ | BbhTemporalSequences242CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 111 OUT412 |
| ✓ | BbhTemporalSequences243CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT491 |
| ✓ | BbhTemporalSequences244CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 142 OUT477 |
| ✓ | BbhTemporalSequences245CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT322 |
| ✓ | BbhTemporalSequences246CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT391 |
| ✓ | BbhTemporalSequences247CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 146 OUT572 |
| ✓ | BbhTemporalSequences248CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 154 OUT588 |
| ✓ | BbhTemporalSequences249CMfind the free time slot from a schedule1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 134 OUT563 |
Reasoning - BBH tracking shuffled objects (3) (set 1)
30 passed |
100.0% |
IN32 047 OUT9 571 | |
| ✓ | BbhTrackingThree10CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT251 |
| ✓ | BbhTrackingThree11CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT429 |
| ✓ | BbhTrackingThree12CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT293 |
| ✓ | BbhTrackingThree13CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT292 |
| ✓ | BbhTrackingThree14CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT248 |
| ✓ | BbhTrackingThree15CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT485 |
| ✓ | BbhTrackingThree16CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT321 |
| ✓ | BbhTrackingThree17CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT252 |
| ✓ | BbhTrackingThree18CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT331 |
| ✓ | BbhTrackingThree19CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT325 |
| ✓ | BbhTrackingThree20CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT510 |
| ✓ | BbhTrackingThree21CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT324 |
| ✓ | BbhTrackingThree22CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT412 |
| ✓ | BbhTrackingThree23CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT251 |
| ✓ | BbhTrackingThree24CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT318 |
| ✓ | BbhTrackingThree25CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT314 |
| ✓ | BbhTrackingThree26CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackingThree27CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT293 |
| ✓ | BbhTrackingThree28CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT330 |
| ✓ | BbhTrackingThree29CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT349 |
| ✓ | BbhTrackingThree30CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT270 |
| ✓ | BbhTrackingThree31CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT359 |
| ✓ | BbhTrackingThree32CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT407 |
| ✓ | BbhTrackingThree33CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT282 |
| ✓ | BbhTrackingThree34CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT322 |
| ✓ | BbhTrackingThree35CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree36CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT283 |
| ✓ | BbhTrackingThree37CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackingThree8CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT274 |
| ✓ | BbhTrackingThree9CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT253 |
Reasoning - BBH tracking shuffled objects (3) (set 2)
30 passed |
100.0% |
IN32 058 OUT8 836 | |
| ✓ | BbhTrackingThree38CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT294 |
| ✓ | BbhTrackingThree39CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT318 |
| ✓ | BbhTrackingThree40CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT254 |
| ✓ | BbhTrackingThree41CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT244 |
| ✓ | BbhTrackingThree42CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT285 |
| ✓ | BbhTrackingThree43CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT251 |
| ✓ | BbhTrackingThree44CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT325 |
| ✓ | BbhTrackingThree45CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackingThree46CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT282 |
| ✓ | BbhTrackingThree47CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT328 |
| ✓ | BbhTrackingThree48CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT324 |
| ✓ | BbhTrackingThree49CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackingThree50CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT322 |
| ✓ | BbhTrackingThree51CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT253 |
| ✓ | BbhTrackingThree52CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT263 |
| ✓ | BbhTrackingThree53CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT321 |
| ✓ | BbhTrackingThree54CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT372 |
| ✓ | BbhTrackingThree55CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT263 |
| ✓ | BbhTrackingThree56CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT262 |
| ✓ | BbhTrackingThree57CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT263 |
| ✓ | BbhTrackingThree58CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT340 |
| ✓ | BbhTrackingThree59CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT330 |
| ✓ | BbhTrackingThree60CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT328 |
| ✓ | BbhTrackingThree61CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT367 |
| ✓ | BbhTrackingThree62CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT338 |
| ✓ | BbhTrackingThree63CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT294 |
| ✓ | BbhTrackingThree64CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT286 |
| ✓ | BbhTrackingThree65CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT251 |
| ✓ | BbhTrackingThree66CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackingThree67CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT284 |
Reasoning - BBH tracking shuffled objects (3) (set 3)
30 passed |
100.0% |
IN32 093 OUT9 684 | |
| ✓ | BbhTrackingThree68CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackingThree69CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT289 |
| ✓ | BbhTrackingThree70CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT263 |
| ✓ | BbhTrackingThree71CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT370 |
| ✓ | BbhTrackingThree72CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT283 |
| ✓ | BbhTrackingThree73CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT317 |
| ✓ | BbhTrackingThree74CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackingThree75CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackingThree76CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT270 |
| ✓ | BbhTrackingThree77CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT346 |
| ✓ | BbhTrackingThree78CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT265 |
| ✓ | BbhTrackingThree79CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT387 |
| ✓ | BbhTrackingThree80CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT426 |
| ✓ | BbhTrackingThree81CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT295 |
| ✓ | BbhTrackingThree82CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT312 |
| ✓ | BbhTrackingThree83CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT341 |
| ✓ | BbhTrackingThree84CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT377 |
| ✓ | BbhTrackingThree85CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT264 |
| ✓ | BbhTrackingThree86CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT337 |
| ✓ | BbhTrackingThree87CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT249 |
| ✓ | BbhTrackingThree88CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT570 |
| ✓ | BbhTrackingThree89CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackingThree90CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT257 |
| ✓ | BbhTrackingThree91CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT303 |
| ✓ | BbhTrackingThree92CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT291 |
| ✓ | BbhTrackingThree93CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT325 |
| ✓ | BbhTrackingThree94CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT254 |
| ✓ | BbhTrackingThree95CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT580 |
| ✓ | BbhTrackingThree96CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT330 |
| ✓ | BbhTrackingThree97CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT322 |
Reasoning - BBH tracking shuffled objects (3) (set 4)
29 passed |
100.0% |
IN30 961 OUT9 002 | |
| ✓ | BbhTrackingThree100CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT283 |
| ✓ | BbhTrackingThree101CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT390 |
| ✓ | BbhTrackingThree102CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT402 |
| ✓ | BbhTrackingThree103CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackingThree104CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT251 |
| ✓ | BbhTrackingThree105CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT386 |
| ✓ | BbhTrackingThree106CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT248 |
| ✓ | BbhTrackingThree107CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackingThree108CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT263 |
| ✓ | BbhTrackingThree109CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT254 |
| ✓ | BbhTrackingThree110CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT282 |
| ✓ | BbhTrackingThree111CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT315 |
| ✓ | BbhTrackingThree112CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT293 |
| ✓ | BbhTrackingThree113CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackingThree114CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT289 |
| ✓ | BbhTrackingThree115CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackingThree116CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT263 |
| ✓ | BbhTrackingThree117CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT422 |
| ✓ | BbhTrackingThree118CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT451 |
| ✓ | BbhTrackingThree119CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT263 |
| ✓ | BbhTrackingThree120CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT433 |
| ✓ | BbhTrackingThree121CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT298 |
| ✓ | BbhTrackingThree122CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT414 |
| ✓ | BbhTrackingThree123CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT250 |
| ✓ | BbhTrackingThree124CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT299 |
| ✓ | BbhTrackingThree125CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT316 |
| ✓ | BbhTrackingThree126CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT311 |
| ✓ | BbhTrackingThree98CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackingThree99CPtrack where 3 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT297 |
Reasoning - BBH tracking shuffled objects (3) (set 5)
32 passed |
100.0% |
IN34 243 OUT9 897 | |
| ✓ | BbhTrackThree127CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT284 |
| ✓ | BbhTrackThree128CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT322 |
| ✓ | BbhTrackThree129CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT320 |
| ✓ | BbhTrackThree130CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT298 |
| ✓ | BbhTrackThree131CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 084 OUT356 |
| ✓ | BbhTrackThree132CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackThree133CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT271 |
| ✓ | BbhTrackThree134CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackThree135CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT372 |
| ✓ | BbhTrackThree136CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT319 |
| ✓ | BbhTrackThree137CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT368 |
| ✓ | BbhTrackThree138CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT249 |
| ✓ | BbhTrackThree139CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT308 |
| ✓ | BbhTrackThree140CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT309 |
| ✓ | BbhTrackThree141CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT289 |
| ✓ | BbhTrackThree142CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT256 |
| ✓ | BbhTrackThree143CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT263 |
| ✓ | BbhTrackThree144CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT347 |
| ✓ | BbhTrackThree145CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT263 |
| ✓ | BbhTrackThree146CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT271 |
| ✓ | BbhTrackThree147CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackThree148CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT290 |
| ✓ | BbhTrackThree149CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 082 OUT350 |
| ✓ | BbhTrackThree150CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackThree151CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT300 |
| ✓ | BbhTrackThree152CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT239 |
| ✓ | BbhTrackThree153CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT263 |
| ✓ | BbhTrackThree154CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT483 |
| ✓ | BbhTrackThree155CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT281 |
| ✓ | BbhTrackThree156CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT544 |
| ✓ | BbhTrackThree157CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT283 |
| ✓ | BbhTrackThree158CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT336 |
Reasoning - BBH tracking shuffled objects (3) (set 6)
32 passed |
100.0% |
IN34 151 OUT9 980 | |
| ✓ | BbhTrackThree159CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT253 |
| ✓ | BbhTrackThree160CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT320 |
| ✓ | BbhTrackThree161CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT313 |
| ✓ | BbhTrackThree162CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT265 |
| ✓ | BbhTrackThree163CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree164CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT262 |
| ✓ | BbhTrackThree165CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT253 |
| ✓ | BbhTrackThree166CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT305 |
| ✓ | BbhTrackThree167CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT428 |
| ✓ | BbhTrackThree168CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT338 |
| ✓ | BbhTrackThree169CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT312 |
| ✓ | BbhTrackThree170CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT294 |
| ✓ | BbhTrackThree171CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT322 |
| ✓ | BbhTrackThree172CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackThree173CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT285 |
| ✓ | BbhTrackThree174CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT324 |
| ✓ | BbhTrackThree175CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT428 |
| ✓ | BbhTrackThree176CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT263 |
| ✓ | BbhTrackThree177CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT281 |
| ✓ | BbhTrackThree178CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT263 |
| ✓ | BbhTrackThree179CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT325 |
| ✓ | BbhTrackThree180CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT461 |
| ✓ | BbhTrackThree181CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackThree182CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT295 |
| ✓ | BbhTrackThree183CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT329 |
| ✓ | BbhTrackThree184CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT366 |
| ✓ | BbhTrackThree185CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT369 |
| ✓ | BbhTrackThree186CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT325 |
| ✓ | BbhTrackThree187CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT321 |
| ✓ | BbhTrackThree188CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT327 |
| ✓ | BbhTrackThree189CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackThree190CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT293 |
Reasoning - BBH tracking shuffled objects (3) (set 7)
32 passed |
100.0% |
IN34 170 OUT9 740 | |
| ✓ | BbhTrackThree191CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT309 |
| ✓ | BbhTrackThree192CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT253 |
| ✓ | BbhTrackThree193CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT313 |
| ✓ | BbhTrackThree194CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT284 |
| ✓ | BbhTrackThree195CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackThree196CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT310 |
| ✓ | BbhTrackThree197CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 065 OUT438 |
| ✓ | BbhTrackThree198CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 074 OUT369 |
| ✓ | BbhTrackThree199CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT387 |
| ✓ | BbhTrackThree200CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT389 |
| ✓ | BbhTrackThree201CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT263 |
| ✓ | BbhTrackThree202CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackThree203CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT243 |
| ✓ | BbhTrackThree204CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT278 |
| ✓ | BbhTrackThree205CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT372 |
| ✓ | BbhTrackThree206CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT277 |
| ✓ | BbhTrackThree207CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT290 |
| ✓ | BbhTrackThree208CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT240 |
| ✓ | BbhTrackThree209CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT263 |
| ✓ | BbhTrackThree210CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT297 |
| ✓ | BbhTrackThree211CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT425 |
| ✓ | BbhTrackThree212CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT254 |
| ✓ | BbhTrackThree213CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT251 |
| ✓ | BbhTrackThree214CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackThree215CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT276 |
| ✓ | BbhTrackThree216CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT395 |
| ✓ | BbhTrackThree217CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT379 |
| ✓ | BbhTrackThree218CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT293 |
| ✓ | BbhTrackThree219CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT290 |
| ✓ | BbhTrackThree220CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT270 |
| ✓ | BbhTrackThree221CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT253 |
| ✓ | BbhTrackThree222CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT283 |
Reasoning - BBH tracking shuffled objects (3) (set 8)
27 passed |
100.0% |
IN28 834 OUT8 933 | |
| ✓ | BbhTrackThree223CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT265 |
| ✓ | BbhTrackThree224CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT400 |
| ✓ | BbhTrackThree225CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT391 |
| ✓ | BbhTrackThree226CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT369 |
| ✓ | BbhTrackThree227CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackThree228CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackThree229CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT254 |
| ✓ | BbhTrackThree230CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT265 |
| ✓ | BbhTrackThree231CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT262 |
| ✓ | BbhTrackThree232CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT263 |
| ✓ | BbhTrackThree233CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT410 |
| ✓ | BbhTrackThree234CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT322 |
| ✓ | BbhTrackThree235CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 072 OUT400 |
| ✓ | BbhTrackThree236CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 067 OUT506 |
| ✓ | BbhTrackThree237CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT380 |
| ✓ | BbhTrackThree238CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 066 OUT295 |
| ✓ | BbhTrackThree239CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT314 |
| ✓ | BbhTrackThree240CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT377 |
| ✓ | BbhTrackThree241CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 070 OUT237 |
| ✓ | BbhTrackThree242CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 063 OUT264 |
| ✓ | BbhTrackThree243CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT408 |
| ✓ | BbhTrackThree244CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 078 OUT333 |
| ✓ | BbhTrackThree245CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 073 OUT266 |
| ✓ | BbhTrackThree246CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 076 OUT343 |
| ✓ | BbhTrackThree247CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 061 OUT482 |
| ✓ | BbhTrackThree248CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT285 |
| ✓ | BbhTrackThree249CQtrack 3 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 068 OUT310 |
Reasoning - BBH tracking shuffled objects (5) (set 1)
33 passed |
100.0% |
IN36 717 OUT18 277 | |
| ✓ | BbhTrackingShuffledFive10CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT526 |
| ✓ | BbhTrackingShuffledFive11CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT469 |
| ✓ | BbhTrackingShuffledFive12CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT614 |
| ✓ | BbhTrackingShuffledFive13CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT531 |
| ✓ | BbhTrackingShuffledFive14CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT979 |
| ✓ | BbhTrackingShuffledFive15CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT460 |
| ✓ | BbhTrackingShuffledFive16CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT423 |
| ✓ | BbhTrackingShuffledFive17CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT752 |
| ✓ | BbhTrackingShuffledFive18CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT577 |
| ✓ | BbhTrackingShuffledFive19CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT435 |
| ✓ | BbhTrackingShuffledFive20CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT532 |
| ✓ | BbhTrackingShuffledFive21CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT555 |
| ✓ | BbhTrackingShuffledFive22CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT542 |
| ✓ | BbhTrackingShuffledFive23CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT487 |
| ✓ | BbhTrackingShuffledFive24CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT586 |
| ✓ | BbhTrackingShuffledFive25CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT529 |
| ✓ | BbhTrackingShuffledFive26CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT500 |
| ✓ | BbhTrackingShuffledFive27CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT530 |
| ✓ | BbhTrackingShuffledFive28CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT456 |
| ✓ | BbhTrackingShuffledFive29CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT671 |
| ✓ | BbhTrackingShuffledFive30CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT593 |
| ✓ | BbhTrackingShuffledFive31CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT555 |
| ✓ | BbhTrackingShuffledFive32CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT512 |
| ✓ | BbhTrackingShuffledFive33CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT779 |
| ✓ | BbhTrackingShuffledFive34CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT461 |
| ✓ | BbhTrackingShuffledFive35CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT533 |
| ✓ | BbhTrackingShuffledFive36CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT471 |
| ✓ | BbhTrackingShuffledFive37CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT573 |
| ✓ | BbhTrackingShuffledFive38CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT545 |
| ✓ | BbhTrackingShuffledFive39CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT592 |
| ✓ | BbhTrackingShuffledFive40CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT439 |
| ✓ | BbhTrackingShuffledFive8CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT583 |
| ✓ | BbhTrackingShuffledFive9CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT487 |
Reasoning - BBH tracking shuffled objects (5) (set 2)
33 passed |
100.0% |
IN36 623 OUT19 966 | |
| ✓ | BbhTrackingShuffledFive41CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT566 |
| ✓ | BbhTrackingShuffledFive42CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT553 |
| ✓ | BbhTrackingShuffledFive43CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT558 |
| ✓ | BbhTrackingShuffledFive44CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT997 |
| ✓ | BbhTrackingShuffledFive45CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT437 |
| ✓ | BbhTrackingShuffledFive46CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT525 |
| ✓ | BbhTrackingShuffledFive47CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT543 |
| ✓ | BbhTrackingShuffledFive48CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 490 |
| ✓ | BbhTrackingShuffledFive49CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT486 |
| ✓ | BbhTrackingShuffledFive50CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT538 |
| ✓ | BbhTrackingShuffledFive51CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT528 |
| ✓ | BbhTrackingShuffledFive52CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT538 |
| ✓ | BbhTrackingShuffledFive53CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT551 |
| ✓ | BbhTrackingShuffledFive54CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT448 |
| ✓ | BbhTrackingShuffledFive55CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT659 |
| ✓ | BbhTrackingShuffledFive56CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT575 |
| ✓ | BbhTrackingShuffledFive57CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT516 |
| ✓ | BbhTrackingShuffledFive58CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 237 |
| ✓ | BbhTrackingShuffledFive59CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT512 |
| ✓ | BbhTrackingShuffledFive60CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT936 |
| ✓ | BbhTrackingShuffledFive61CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT484 |
| ✓ | BbhTrackingShuffledFive62CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT423 |
| ✓ | BbhTrackingShuffledFive63CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT505 |
| ✓ | BbhTrackingShuffledFive64CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT444 |
| ✓ | BbhTrackingShuffledFive65CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT591 |
| ✓ | BbhTrackingShuffledFive66CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT552 |
| ✓ | BbhTrackingShuffledFive67CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT647 |
| ✓ | BbhTrackingShuffledFive68CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT470 |
| ✓ | BbhTrackingShuffledFive69CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT710 |
| ✓ | BbhTrackingShuffledFive70CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT550 |
| ✓ | BbhTrackingShuffledFive71CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT471 |
| ✓ | BbhTrackingShuffledFive72CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT425 |
| ✓ | BbhTrackingShuffledFive73CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT501 |
Reasoning - BBH tracking shuffled objects (5) (set 3)
32 passed |
100.0% |
IN35 590 OUT19 286 | |
| ✓ | BbhTrackingShuffledFive100CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT556 |
| ✓ | BbhTrackingShuffledFive101CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT902 |
| ✓ | BbhTrackingShuffledFive102CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT414 |
| ✓ | BbhTrackingShuffledFive103CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT466 |
| ✓ | BbhTrackingShuffledFive104CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT478 |
| ✓ | BbhTrackingShuffledFive105CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT423 |
| ✓ | BbhTrackingShuffledFive74CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT591 |
| ✓ | BbhTrackingShuffledFive75CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT556 |
| ✓ | BbhTrackingShuffledFive76CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT470 |
| ✓ | BbhTrackingShuffledFive77CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT469 |
| ✓ | BbhTrackingShuffledFive78CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT802 |
| ✓ | BbhTrackingShuffledFive79CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT763 |
| ✓ | BbhTrackingShuffledFive80CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT547 |
| ✓ | BbhTrackingShuffledFive81CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT520 |
| ✓ | BbhTrackingShuffledFive82CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT856 |
| ✓ | BbhTrackingShuffledFive83CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT487 |
| ✓ | BbhTrackingShuffledFive84CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT739 |
| ✓ | BbhTrackingShuffledFive85CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT582 |
| ✓ | BbhTrackingShuffledFive86CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT573 |
| ✓ | BbhTrackingShuffledFive87CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT515 |
| ✓ | BbhTrackingShuffledFive88CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT760 |
| ✓ | BbhTrackingShuffledFive89CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT444 |
| ✓ | BbhTrackingShuffledFive90CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 167 |
| ✓ | BbhTrackingShuffledFive91CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT482 |
| ✓ | BbhTrackingShuffledFive92CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT516 |
| ✓ | BbhTrackingShuffledFive93CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT934 |
| ✓ | BbhTrackingShuffledFive94CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT679 |
| ✓ | BbhTrackingShuffledFive95CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT472 |
| ✓ | BbhTrackingShuffledFive96CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT459 |
| ✓ | BbhTrackingShuffledFive97CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT611 |
| ✓ | BbhTrackingShuffledFive98CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT522 |
| ✓ | BbhTrackingShuffledFive99CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT531 |
Reasoning - BBH tracking shuffled objects (5) (set 4)
31 passed |
100.0% |
IN34 491 OUT17 839 | |
| ✓ | BbhTrackingShuffledFive106CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT581 |
| ✓ | BbhTrackingShuffledFive107CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT646 |
| ✓ | BbhTrackingShuffledFive108CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT469 |
| ✓ | BbhTrackingShuffledFive109CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT659 |
| ✓ | BbhTrackingShuffledFive110CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT468 |
| ✓ | BbhTrackingShuffledFive111CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT513 |
| ✓ | BbhTrackingShuffledFive112CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 158 |
| ✓ | BbhTrackingShuffledFive113CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT999 |
| ✓ | BbhTrackingShuffledFive114CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT621 |
| ✓ | BbhTrackingShuffledFive115CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 130 OUT660 |
| ✓ | BbhTrackingShuffledFive116CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT599 |
| ✓ | BbhTrackingShuffledFive117CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT676 |
| ✓ | BbhTrackingShuffledFive118CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT468 |
| ✓ | BbhTrackingShuffledFive119CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT468 |
| ✓ | BbhTrackingShuffledFive120CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT494 |
| ✓ | BbhTrackingShuffledFive121CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT923 |
| ✓ | BbhTrackingShuffledFive122CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT542 |
| ✓ | BbhTrackingShuffledFive123CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT471 |
| ✓ | BbhTrackingShuffledFive124CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT472 |
| ✓ | BbhTrackingShuffledFive125CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT440 |
| ✓ | BbhTrackingShuffledFive126CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT591 |
| ✓ | BbhTrackingShuffledFive127CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT575 |
| ✓ | BbhTrackingShuffledFive128CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT473 |
| ✓ | BbhTrackingShuffledFive129CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT459 |
| ✓ | BbhTrackingShuffledFive130CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT500 |
| ✓ | BbhTrackingShuffledFive131CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT437 |
| ✓ | BbhTrackingShuffledFive132CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT466 |
| ✓ | BbhTrackingShuffledFive133CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT508 |
| ✓ | BbhTrackingShuffledFive134CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT466 |
| ✓ | BbhTrackingShuffledFive135CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT519 |
| ✓ | BbhTrackingShuffledFive136CMtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 116 OUT518 |
Reasoning - BBH tracking shuffled objects (5) (set 5)
30 passed2 failed |
93.8% |
IN35 563 OUT17 946 | |
| ✓ | BbhTrackFive137CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT513 |
| ✓ | BbhTrackFive138CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT591 |
| ✓ | BbhTrackFive139CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT563 |
| ✓ | BbhTrackFive140CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT796 |
| ✓ | BbhTrackFive141CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT539 |
| ✓ | BbhTrackFive142CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT471 |
| ✓ | BbhTrackFive143CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT537 |
| ✓ | BbhTrackFive144CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT994 |
| ✓ | BbhTrackFive145CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT450 |
| ✓ | BbhTrackFive146CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT719 |
| ✓ | BbhTrackFive147CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT667 |
| ✓ | BbhTrackFive148CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT545 |
| ✓ | BbhTrackFive149CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT457 |
| ✓ | BbhTrackFive150CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT445 |
| ✓ | BbhTrackFive151CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT595 |
| ✓ | BbhTrackFive152CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT514 |
| ✓ | BbhTrackFive153CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT645 |
| ✓ | BbhTrackFive154CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT558 |
| ✓ | BbhTrackFive155CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT568 |
| × | BbhTrackFive156CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("")
Context: BbhTrackFive156CN: expected , got [(E)] |
Failed | IN1 105 OUT457 |
| × | BbhTrackFive157CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("")
Context: BbhTrackFive157CN: expected , got [(A)] |
Failed | IN1 117 OUT429 |
| ✓ | BbhTrackFive158CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT480 |
| ✓ | BbhTrackFive159CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT470 |
| ✓ | BbhTrackFive160CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT468 |
| ✓ | BbhTrackFive161CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT789 |
| ✓ | BbhTrackFive162CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT467 |
| ✓ | BbhTrackFive163CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT457 |
| ✓ | BbhTrackFive164CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT555 |
| ✓ | BbhTrackFive165CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT514 |
| ✓ | BbhTrackFive166CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT664 |
| ✓ | BbhTrackFive167CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT523 |
| ✓ | BbhTrackFive168CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT506 |
Reasoning - BBH tracking shuffled objects (5) (set 6)
32 passed |
100.0% |
IN35 607 OUT18 162 | |
| ✓ | BbhTrackFive169CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT545 |
| ✓ | BbhTrackFive170CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT509 |
| ✓ | BbhTrackFive171CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT555 |
| ✓ | BbhTrackFive172CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT957 |
| ✓ | BbhTrackFive173CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT556 |
| ✓ | BbhTrackFive174CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT413 |
| ✓ | BbhTrackFive175CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT468 |
| ✓ | BbhTrackFive176CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 118 OUT543 |
| ✓ | BbhTrackFive177CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT482 |
| ✓ | BbhTrackFive178CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT444 |
| ✓ | BbhTrackFive179CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 126 OUT629 |
| ✓ | BbhTrackFive180CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT500 |
| ✓ | BbhTrackFive181CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT527 |
| ✓ | BbhTrackFive182CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT553 |
| ✓ | BbhTrackFive183CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 053 |
| ✓ | BbhTrackFive184CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT541 |
| ✓ | BbhTrackFive185CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT583 |
| ✓ | BbhTrackFive186CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT1 096 |
| ✓ | BbhTrackFive187CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT487 |
| ✓ | BbhTrackFive188CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT619 |
| ✓ | BbhTrackFive189CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT436 |
| ✓ | BbhTrackFive190CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT566 |
| ✓ | BbhTrackFive191CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT482 |
| ✓ | BbhTrackFive192CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT521 |
| ✓ | BbhTrackFive193CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT474 |
| ✓ | BbhTrackFive194CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT613 |
| ✓ | BbhTrackFive195CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT530 |
| ✓ | BbhTrackFive196CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT518 |
| ✓ | BbhTrackFive197CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT471 |
| ✓ | BbhTrackFive198CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT466 |
| ✓ | BbhTrackFive199CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT540 |
| ✓ | BbhTrackFive200CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT485 |
Reasoning - BBH tracking shuffled objects (5) (set 7)
32 passed |
100.0% |
IN35 568 OUT19 445 | |
| ✓ | BbhTrackFive201CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT427 |
| ✓ | BbhTrackFive202CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT483 |
| ✓ | BbhTrackFive203CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT625 |
| ✓ | BbhTrackFive204CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT481 |
| ✓ | BbhTrackFive205CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT442 |
| ✓ | BbhTrackFive206CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT468 |
| ✓ | BbhTrackFive207CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 146 |
| ✓ | BbhTrackFive208CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT669 |
| ✓ | BbhTrackFive209CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT649 |
| ✓ | BbhTrackFive210CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT491 |
| ✓ | BbhTrackFive211CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 109 OUT579 |
| ✓ | BbhTrackFive212CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT471 |
| ✓ | BbhTrackFive213CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 108 OUT564 |
| ✓ | BbhTrackFive214CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT571 |
| ✓ | BbhTrackFive215CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT474 |
| ✓ | BbhTrackFive216CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 110 |
| ✓ | BbhTrackFive217CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT869 |
| ✓ | BbhTrackFive218CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT640 |
| ✓ | BbhTrackFive219CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT565 |
| ✓ | BbhTrackFive220CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT792 |
| ✓ | BbhTrackFive221CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT481 |
| ✓ | BbhTrackFive222CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT431 |
| ✓ | BbhTrackFive223CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT503 |
| ✓ | BbhTrackFive224CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT482 |
| ✓ | BbhTrackFive225CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT673 |
| ✓ | BbhTrackFive226CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 122 OUT558 |
| ✓ | BbhTrackFive227CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT481 |
| ✓ | BbhTrackFive228CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT611 |
| ✓ | BbhTrackFive229CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 112 OUT533 |
| ✓ | BbhTrackFive230CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT506 |
| ✓ | BbhTrackFive231CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT1 080 |
| ✓ | BbhTrackFive232CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 110 OUT590 |
Reasoning - BBH tracking shuffled objects (5) (set 8)
32 passed |
100.0% |
IN36 272 OUT22 019 | |
| ✓ | BbhTrackFive233CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT489 |
| ✓ | BbhTrackFive234CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 132 OUT578 |
| ✓ | BbhTrackFive235CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT987 |
| ✓ | BbhTrackFive236CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT470 |
| ✓ | BbhTrackFive237CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT443 |
| ✓ | BbhTrackFive238CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT439 |
| ✓ | BbhTrackFive239CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 120 OUT567 |
| ✓ | BbhTrackFive240CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 117 OUT517 |
| ✓ | BbhTrackFive241CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 128 OUT562 |
| ✓ | BbhTrackFive242CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT805 |
| ✓ | BbhTrackFive243CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 101 OUT585 |
| ✓ | BbhTrackFive244CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 124 OUT557 |
| ✓ | BbhTrackFive245CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 105 OUT505 |
| ✓ | BbhTrackFive246CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 114 OUT511 |
| ✓ | BbhTrackFive247CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT1 043 |
| ✓ | BbhTrackFive248CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 103 OUT558 |
| ✓ | BbhTrackFive249CNtrack 5 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 107 OUT662 |
| ✓ | BbhTrackSeven10CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT889 |
| ✓ | BbhTrackSeven11CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT910 |
| ✓ | BbhTrackSeven12CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT896 |
| ✓ | BbhTrackSeven13CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT775 |
| ✓ | BbhTrackSeven14CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT931 |
| ✓ | BbhTrackSeven15CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT699 |
| ✓ | BbhTrackSeven16CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT470 |
| ✓ | BbhTrackSeven17CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 146 |
| ✓ | BbhTrackSeven18CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT509 |
| ✓ | BbhTrackSeven19CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT1 222 |
| ✓ | BbhTrackSeven20CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT843 |
| ✓ | BbhTrackSeven21CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT595 |
| ✓ | BbhTrackSeven22CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT545 |
| ✓ | BbhTrackSeven8CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT512 |
| ✓ | BbhTrackSeven9CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT799 |
Reasoning - BBH tracking shuffled objects (7) (set 1)
30 passed |
100.0% |
IN34 796 OUT23 749 | |
| ✓ | BbhTrackSeven23CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT764 |
| ✓ | BbhTrackSeven24CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT822 |
| ✓ | BbhTrackSeven25CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT757 |
| ✓ | BbhTrackSeven26CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT703 |
| ✓ | BbhTrackSeven27CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT772 |
| ✓ | BbhTrackSeven28CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT767 |
| ✓ | BbhTrackSeven29CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT758 |
| ✓ | BbhTrackSeven30CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT840 |
| ✓ | BbhTrackSeven31CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT692 |
| ✓ | BbhTrackSeven32CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT884 |
| ✓ | BbhTrackSeven33CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT727 |
| ✓ | BbhTrackSeven34CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT782 |
| ✓ | BbhTrackSeven35CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT733 |
| ✓ | BbhTrackSeven36CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT874 |
| ✓ | BbhTrackSeven37CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT762 |
| ✓ | BbhTrackSeven38CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT799 |
| ✓ | BbhTrackSeven39CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT938 |
| ✓ | BbhTrackSeven40CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT778 |
| ✓ | BbhTrackSeven41CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT826 |
| ✓ | BbhTrackSeven42CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT829 |
| ✓ | BbhTrackSeven43CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT762 |
| ✓ | BbhTrackSeven44CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT791 |
| ✓ | BbhTrackSeven45CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT730 |
| ✓ | BbhTrackSeven46CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT444 |
| ✓ | BbhTrackSeven47CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 182 OUT931 |
| ✓ | BbhTrackSeven48CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT860 |
| ✓ | BbhTrackSeven49CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT835 |
| ✓ | BbhTrackSeven50CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 133 |
| ✓ | BbhTrackSeven51CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 175 OUT810 |
| ✓ | BbhTrackSeven52CNtrack 7 objects' positions after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT646 |
Reasoning - BBH tracking shuffled objects (7) (set 2)
32 passed |
100.0% |
IN37 146 OUT26 933 | |
| ✓ | BbhTrackingSeven53COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT773 |
| ✓ | BbhTrackingSeven54COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 178 OUT889 |
| ✓ | BbhTrackingSeven55COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT933 |
| ✓ | BbhTrackingSeven56COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT795 |
| ✓ | BbhTrackingSeven57COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT909 |
| ✓ | BbhTrackingSeven58COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT839 |
| ✓ | BbhTrackingSeven59COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 176 OUT897 |
| ✓ | BbhTrackingSeven60COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT834 |
| ✓ | BbhTrackingSeven61COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT900 |
| ✓ | BbhTrackingSeven62COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT971 |
| ✓ | BbhTrackingSeven63COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT747 |
| ✓ | BbhTrackingSeven64COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT523 |
| ✓ | BbhTrackingSeven65COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT770 |
| ✓ | BbhTrackingSeven66COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT827 |
| ✓ | BbhTrackingSeven67COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT819 |
| ✓ | BbhTrackingSeven68COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 041 |
| ✓ | BbhTrackingSeven69COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT781 |
| ✓ | BbhTrackingSeven70COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT708 |
| ✓ | BbhTrackingSeven71COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT714 |
| ✓ | BbhTrackingSeven72COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT958 |
| ✓ | BbhTrackingSeven73COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 170 OUT746 |
| ✓ | BbhTrackingSeven74COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 357 |
| ✓ | BbhTrackingSeven75COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT776 |
| ✓ | BbhTrackingSeven76COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT953 |
| ✓ | BbhTrackingSeven77COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT563 |
| ✓ | BbhTrackingSeven78COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT884 |
| ✓ | BbhTrackingSeven79COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT756 |
| ✓ | BbhTrackingSeven80COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT843 |
| ✓ | BbhTrackingSeven81COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT925 |
| ✓ | BbhTrackingSeven82COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT886 |
| ✓ | BbhTrackingSeven83COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT831 |
| ✓ | BbhTrackingSeven84COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT785 |
Reasoning - BBH tracking shuffled objects (7) (set 3)
32 passed |
100.0% |
IN37 233 OUT23 918 | |
| ✓ | BbhTrackingSeven100COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT805 |
| ✓ | BbhTrackingSeven101COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 186 OUT916 |
| ✓ | BbhTrackingSeven102COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT512 |
| ✓ | BbhTrackingSeven103COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 168 OUT863 |
| ✓ | BbhTrackingSeven104COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT997 |
| ✓ | BbhTrackingSeven105COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT830 |
| ✓ | BbhTrackingSeven106COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT818 |
| ✓ | BbhTrackingSeven107COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT440 |
| ✓ | BbhTrackingSeven108COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT776 |
| ✓ | BbhTrackingSeven109COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 176 OUT622 |
| ✓ | BbhTrackingSeven110COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT833 |
| ✓ | BbhTrackingSeven111COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT898 |
| ✓ | BbhTrackingSeven112COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT790 |
| ✓ | BbhTrackingSeven113COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT782 |
| ✓ | BbhTrackingSeven114COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT790 |
| ✓ | BbhTrackingSeven115COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT561 |
| ✓ | BbhTrackingSeven116COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT843 |
| ✓ | BbhTrackingSeven85COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT846 |
| ✓ | BbhTrackingSeven86COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 168 OUT858 |
| ✓ | BbhTrackingSeven87COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT472 |
| ✓ | BbhTrackingSeven88COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT705 |
| ✓ | BbhTrackingSeven89COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT732 |
| ✓ | BbhTrackingSeven90COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT541 |
| ✓ | BbhTrackingSeven91COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT856 |
| ✓ | BbhTrackingSeven92COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT896 |
| ✓ | BbhTrackingSeven93COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT571 |
| ✓ | BbhTrackingSeven94COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT919 |
| ✓ | BbhTrackingSeven95COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT724 |
| ✓ | BbhTrackingSeven96COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT460 |
| ✓ | BbhTrackingSeven97COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT888 |
| ✓ | BbhTrackingSeven98COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT764 |
| ✓ | BbhTrackingSeven99COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT610 |
Reasoning - BBH tracking shuffled objects (7) (set 4)
32 passed |
100.0% |
IN37 229 OUT26 947 | |
| ✓ | BbhTrackingSeven117COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT883 |
| ✓ | BbhTrackingSeven118COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT846 |
| ✓ | BbhTrackingSeven119COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT843 |
| ✓ | BbhTrackingSeven120COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT874 |
| ✓ | BbhTrackingSeven121COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 504 |
| ✓ | BbhTrackingSeven122COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 003 |
| ✓ | BbhTrackingSeven123COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT766 |
| ✓ | BbhTrackingSeven124COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 170 OUT870 |
| ✓ | BbhTrackingSeven125COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT959 |
| ✓ | BbhTrackingSeven126COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT840 |
| ✓ | BbhTrackingSeven127COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT597 |
| ✓ | BbhTrackingSeven128COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 180 OUT932 |
| ✓ | BbhTrackingSeven129COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 156 OUT482 |
| ✓ | BbhTrackingSeven130COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 182 OUT925 |
| ✓ | BbhTrackingSeven131COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 176 OUT905 |
| ✓ | BbhTrackingSeven132COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT888 |
| ✓ | BbhTrackingSeven133COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT440 |
| ✓ | BbhTrackingSeven134COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT986 |
| ✓ | BbhTrackingSeven135COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT800 |
| ✓ | BbhTrackingSeven136COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT729 |
| ✓ | BbhTrackingSeven137COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT898 |
| ✓ | BbhTrackingSeven138COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT840 |
| ✓ | BbhTrackingSeven139COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 184 OUT972 |
| ✓ | BbhTrackingSeven140COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 176 OUT594 |
| ✓ | BbhTrackingSeven141COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT806 |
| ✓ | BbhTrackingSeven142COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 176 OUT889 |
| ✓ | BbhTrackingSeven143COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT902 |
| ✓ | BbhTrackingSeven144COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT879 |
| ✓ | BbhTrackingSeven145COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 060 |
| ✓ | BbhTrackingSeven146COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT449 |
| ✓ | BbhTrackingSeven147COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT746 |
| ✓ | BbhTrackingSeven148COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT840 |
Reasoning - BBH tracking shuffled objects (7) (set 5)
31 passed |
100.0% |
IN36 047 OUT26 070 | |
| ✓ | BbhTrackingSeven149COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT483 |
| ✓ | BbhTrackingSeven150COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT911 |
| ✓ | BbhTrackingSeven151COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT860 |
| ✓ | BbhTrackingSeven152COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 448 |
| ✓ | BbhTrackingSeven153COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT887 |
| ✓ | BbhTrackingSeven154COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 168 OUT862 |
| ✓ | BbhTrackingSeven155COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT894 |
| ✓ | BbhTrackingSeven156COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 176 OUT586 |
| ✓ | BbhTrackingSeven157COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT538 |
| ✓ | BbhTrackingSeven158COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT1 748 |
| ✓ | BbhTrackingSeven159COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT762 |
| ✓ | BbhTrackingSeven160COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 605 |
| ✓ | BbhTrackingSeven161COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT813 |
| ✓ | BbhTrackingSeven162COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT836 |
| ✓ | BbhTrackingSeven163COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT769 |
| ✓ | BbhTrackingSeven164COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT837 |
| ✓ | BbhTrackingSeven165COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT717 |
| ✓ | BbhTrackingSeven166COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT833 |
| ✓ | BbhTrackingSeven167COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 164 OUT899 |
| ✓ | BbhTrackingSeven168COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT804 |
| ✓ | BbhTrackingSeven169COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT480 |
| ✓ | BbhTrackingSeven170COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT651 |
| ✓ | BbhTrackingSeven171COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT895 |
| ✓ | BbhTrackingSeven172COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 182 OUT617 |
| ✓ | BbhTrackingSeven173COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT777 |
| ✓ | BbhTrackingSeven174COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT854 |
| ✓ | BbhTrackingSeven175COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT763 |
| ✓ | BbhTrackingSeven176COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 170 OUT925 |
| ✓ | BbhTrackingSeven177COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT803 |
| ✓ | BbhTrackingSeven178COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT721 |
| ✓ | BbhTrackingSeven179COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT492 |
Reasoning - BBH tracking shuffled objects (7) (set 6)
31 passed |
100.0% |
IN36 055 OUT24 964 | |
| ✓ | BbhTrackingSeven180COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT903 |
| ✓ | BbhTrackingSeven181COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT751 |
| ✓ | BbhTrackingSeven182COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT896 |
| ✓ | BbhTrackingSeven183COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT795 |
| ✓ | BbhTrackingSeven184COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT806 |
| ✓ | BbhTrackingSeven185COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT827 |
| ✓ | BbhTrackingSeven186COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT762 |
| ✓ | BbhTrackingSeven187COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT938 |
| ✓ | BbhTrackingSeven188COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT801 |
| ✓ | BbhTrackingSeven189COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT831 |
| ✓ | BbhTrackingSeven190COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT838 |
| ✓ | BbhTrackingSeven191COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT769 |
| ✓ | BbhTrackingSeven192COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT764 |
| ✓ | BbhTrackingSeven193COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT631 |
| ✓ | BbhTrackingSeven194COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT853 |
| ✓ | BbhTrackingSeven195COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT598 |
| ✓ | BbhTrackingSeven196COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT853 |
| ✓ | BbhTrackingSeven197COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT684 |
| ✓ | BbhTrackingSeven198COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT782 |
| ✓ | BbhTrackingSeven199COtrack each player's position after a series of position swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT539 |
| ✓ | BbhTrackingSeven200COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT876 |
| ✓ | BbhTrackingSeven201COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT821 |
| ✓ | BbhTrackingSeven202COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT727 |
| ✓ | BbhTrackingSeven203COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT924 |
| ✓ | BbhTrackingSeven204COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT745 |
| ✓ | BbhTrackingSeven205COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT882 |
| ✓ | BbhTrackingSeven206COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT889 |
| ✓ | BbhTrackingSeven207COtrack who ends up with which ball after seven players trade balls1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT751 |
| ✓ | BbhTrackingSeven208COtrack each dancer's partner after a series of partner switches1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT864 |
| ✓ | BbhTrackingSeven209COtrack who ends up with which book after seven friends trade books1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 182 OUT996 |
| ✓ | BbhTrackingSeven210COtrack who ends up with which present after a series of gift swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT868 |
Reasoning - BBH tracking shuffled objects (7) (set 7)
39 passed |
100.0% |
IN45 365 OUT34 269 | |
| ✓ | BbhTrackingSeven211CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT881 |
| ✓ | BbhTrackingSeven212CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT876 |
| ✓ | BbhTrackingSeven213CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT823 |
| ✓ | BbhTrackingSeven214CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT775 |
| ✓ | BbhTrackingSeven215CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 159 OUT733 |
| ✓ | BbhTrackingSeven216CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT873 |
| ✓ | BbhTrackingSeven217CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 170 OUT703 |
| ✓ | BbhTrackingSeven218CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT831 |
| ✓ | BbhTrackingSeven219CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT912 |
| ✓ | BbhTrackingSeven220CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT842 |
| ✓ | BbhTrackingSeven221CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 149 OUT991 |
| ✓ | BbhTrackingSeven222CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT840 |
| ✓ | BbhTrackingSeven223CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT1 195 |
| ✓ | BbhTrackingSeven224CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT857 |
| ✓ | BbhTrackingSeven225CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 172 OUT876 |
| ✓ | BbhTrackingSeven226CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 162 OUT880 |
| ✓ | BbhTrackingSeven227CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT1 642 |
| ✓ | BbhTrackingSeven228CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT609 |
| ✓ | BbhTrackingSeven229CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT852 |
| ✓ | BbhTrackingSeven230CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 153 OUT829 |
| ✓ | BbhTrackingSeven231CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 182 OUT919 |
| ✓ | BbhTrackingSeven232CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT764 |
| ✓ | BbhTrackingSeven233CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT759 |
| ✓ | BbhTrackingSeven234CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 052 |
| ✓ | BbhTrackingSeven235CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT805 |
| ✓ | BbhTrackingSeven236CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT917 |
| ✓ | BbhTrackingSeven237CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 155 OUT762 |
| ✓ | BbhTrackingSeven238CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 160 OUT637 |
| ✓ | BbhTrackingSeven239CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 173 OUT738 |
| ✓ | BbhTrackingSeven240CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 158 OUT867 |
| ✓ | BbhTrackingSeven241CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 182 OUT925 |
| ✓ | BbhTrackingSeven242CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 151 OUT686 |
| ✓ | BbhTrackingSeven243CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 169 OUT646 |
| ✓ | BbhTrackingSeven244CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 171 OUT832 |
| ✓ | BbhTrackingSeven245CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT889 |
| ✓ | BbhTrackingSeven246CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 157 OUT1 680 |
| ✓ | BbhTrackingSeven247CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 174 OUT892 |
| ✓ | BbhTrackingSeven248CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 166 OUT910 |
| ✓ | BbhTrackingSeven249CPtrack where 7 objects end up after a sequence of swaps1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 167 OUT769 |
Reasoning - BBH web of lies (set 1)
35 passed |
100.0% |
IN34 992 OUT10 759 | |
| ✓ | BbhWebOfLies10CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT337 |
| ✓ | BbhWebOfLies11CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT372 |
| ✓ | BbhWebOfLies12CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT245 |
| ✓ | BbhWebOfLies13CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT319 |
| ✓ | BbhWebOfLies14CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT327 |
| ✓ | BbhWebOfLies15CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT327 |
| ✓ | BbhWebOfLies16CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT320 |
| ✓ | BbhWebOfLies17CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT316 |
| ✓ | BbhWebOfLies18CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT242 |
| ✓ | BbhWebOfLies19CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT490 |
| ✓ | BbhWebOfLies20CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT286 |
| ✓ | BbhWebOfLies21CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT228 |
| ✓ | BbhWebOfLies22CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT323 |
| ✓ | BbhWebOfLies23CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT248 |
| ✓ | BbhWebOfLies24CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT316 |
| ✓ | BbhWebOfLies25CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT239 |
| ✓ | BbhWebOfLies26CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT271 |
| ✓ | BbhWebOfLies27CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT352 |
| ✓ | BbhWebOfLies28CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT326 |
| ✓ | BbhWebOfLies29CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT475 |
| ✓ | BbhWebOfLies30CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT288 |
| ✓ | BbhWebOfLies31CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT326 |
| ✓ | BbhWebOfLies32CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT340 |
| ✓ | BbhWebOfLies33CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT271 |
| ✓ | BbhWebOfLies34CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT318 |
| ✓ | BbhWebOfLies35CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT252 |
| ✓ | BbhWebOfLies36CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT309 |
| ✓ | BbhWebOfLies37CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT231 |
| ✓ | BbhWebOfLies38CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT339 |
| ✓ | BbhWebOfLies39CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT302 |
| ✓ | BbhWebOfLies40CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT319 |
| ✓ | BbhWebOfLies41CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT231 |
| ✓ | BbhWebOfLies42CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT328 |
| ✓ | BbhWebOfLies8CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT223 |
| ✓ | BbhWebOfLies9CQevaluate a chain of truth-tellers and liars1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT323 |
Reasoning - BBH web of lies (set 2)
32 passed |
100.0% |
IN31 992 OUT9 552 | |
| ✓ | BbhWebOfLies43CRtruth-teller/liar chain — Does Audrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT237 |
| ✓ | BbhWebOfLies44CRtruth-teller/liar chain — Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT358 |
| ✓ | BbhWebOfLies45CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT309 |
| ✓ | BbhWebOfLies46CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT231 |
| ✓ | BbhWebOfLies47CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT343 |
| ✓ | BbhWebOfLies48CRtruth-teller/liar chain — Does Maybelle tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT220 |
| ✓ | BbhWebOfLies49CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT317 |
| ✓ | BbhWebOfLies50CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT262 |
| ✓ | BbhWebOfLies51CRtruth-teller/liar chain — Does Sherrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT279 |
| ✓ | BbhWebOfLies52CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT331 |
| ✓ | BbhWebOfLies53CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT228 |
| ✓ | BbhWebOfLies54CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT308 |
| ✓ | BbhWebOfLies55CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT270 |
| ✓ | BbhWebOfLies56CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT286 |
| ✓ | BbhWebOfLies57CRtruth-teller/liar chain — Does Shalonda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT294 |
| ✓ | BbhWebOfLies58CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT319 |
| ✓ | BbhWebOfLies59CRtruth-teller/liar chain — Does Crista tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT339 |
| ✓ | BbhWebOfLies60CRtruth-teller/liar chain — Does Sima tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT293 |
| ✓ | BbhWebOfLies61CRtruth-teller/liar chain — Does Helene tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT320 |
| ✓ | BbhWebOfLies62CRtruth-teller/liar chain — Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT313 |
| ✓ | BbhWebOfLies63CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT242 |
| ✓ | BbhWebOfLies64CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT378 |
| ✓ | BbhWebOfLies65CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT336 |
| ✓ | BbhWebOfLies66CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT303 |
| ✓ | BbhWebOfLies67CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT342 |
| ✓ | BbhWebOfLies68CRtruth-teller/liar chain — Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT341 |
| ✓ | BbhWebOfLies69CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT282 |
| ✓ | BbhWebOfLies70CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT330 |
| ✓ | BbhWebOfLies71CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT353 |
| ✓ | BbhWebOfLies72CRtruth-teller/liar chain — Does Vernell tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT297 |
| ✓ | BbhWebOfLies73CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT258 |
| ✓ | BbhWebOfLies74CRtruth-teller/liar chain — Does Jaymie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT233 |
Reasoning - BBH web of lies (set 3)
32 passed |
100.0% |
IN31 996 OUT9 675 | |
| ✓ | BbhWebOfLies100CRtruth-teller/liar chain — Does Fidel tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT304 |
| ✓ | BbhWebOfLies101CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT324 |
| ✓ | BbhWebOfLies102CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT301 |
| ✓ | BbhWebOfLies103CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT292 |
| ✓ | BbhWebOfLies104CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT340 |
| ✓ | BbhWebOfLies105CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT330 |
| ✓ | BbhWebOfLies106CRtruth-teller/liar chain — Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT338 |
| ✓ | BbhWebOfLies75CRtruth-teller/liar chain — Does Crista tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT248 |
| ✓ | BbhWebOfLies76CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT277 |
| ✓ | BbhWebOfLies77CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT252 |
| ✓ | BbhWebOfLies78CRtruth-teller/liar chain — Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT343 |
| ✓ | BbhWebOfLies79CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT353 |
| ✓ | BbhWebOfLies80CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT312 |
| ✓ | BbhWebOfLies81CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT332 |
| ✓ | BbhWebOfLies82CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT342 |
| ✓ | BbhWebOfLies83CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT274 |
| ✓ | BbhWebOfLies84CRtruth-teller/liar chain — Does Michael tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT305 |
| ✓ | BbhWebOfLies85CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT291 |
| ✓ | BbhWebOfLies86CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT297 |
| ✓ | BbhWebOfLies87CRtruth-teller/liar chain — Does Leda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT372 |
| ✓ | BbhWebOfLies88CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT310 |
| ✓ | BbhWebOfLies89CRtruth-teller/liar chain — Does Fletcher tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT306 |
| ✓ | BbhWebOfLies90CRtruth-teller/liar chain — Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 008 OUT341 |
| ✓ | BbhWebOfLies91CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT242 |
| ✓ | BbhWebOfLies92CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT250 |
| ✓ | BbhWebOfLies93CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT294 |
| ✓ | BbhWebOfLies94CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT303 |
| ✓ | BbhWebOfLies95CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT262 |
| ✓ | BbhWebOfLies96CRtruth-teller/liar chain — Does Phoebe tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT247 |
| ✓ | BbhWebOfLies97CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT321 |
| ✓ | BbhWebOfLies98CRtruth-teller/liar chain — Does Leda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT283 |
| ✓ | BbhWebOfLies99CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT289 |
Reasoning - BBH web of lies (set 4)
32 passed |
100.0% |
IN32 016 OUT9 869 | |
| ✓ | BbhWebOfLies107CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT226 |
| ✓ | BbhWebOfLies108CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT296 |
| ✓ | BbhWebOfLies109CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT232 |
| ✓ | BbhWebOfLies110CRtruth-teller/liar chain — Does Kristian tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT362 |
| ✓ | BbhWebOfLies111CRtruth-teller/liar chain — Does Crista tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT322 |
| ✓ | BbhWebOfLies112CRtruth-teller/liar chain — Does Bernita tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT332 |
| ✓ | BbhWebOfLies113CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT221 |
| ✓ | BbhWebOfLies114CRtruth-teller/liar chain — Does Millie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT420 |
| ✓ | BbhWebOfLies115CRtruth-teller/liar chain — Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT230 |
| ✓ | BbhWebOfLies116CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT417 |
| ✓ | BbhWebOfLies117CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT328 |
| ✓ | BbhWebOfLies118CRtruth-teller/liar chain — Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT330 |
| ✓ | BbhWebOfLies119CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT242 |
| ✓ | BbhWebOfLies120CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT302 |
| ✓ | BbhWebOfLies121CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT278 |
| ✓ | BbhWebOfLies122CRtruth-teller/liar chain — Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT347 |
| ✓ | BbhWebOfLies123CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT364 |
| ✓ | BbhWebOfLies124CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT331 |
| ✓ | BbhWebOfLies125CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT307 |
| ✓ | BbhWebOfLies126CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT331 |
| ✓ | BbhWebOfLies127CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT261 |
| ✓ | BbhWebOfLies128CRtruth-teller/liar chain — Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT242 |
| ✓ | BbhWebOfLies129CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT258 |
| ✓ | BbhWebOfLies130CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT332 |
| ✓ | BbhWebOfLies131CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT306 |
| ✓ | BbhWebOfLies132CRtruth-teller/liar chain — Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT319 |
| ✓ | BbhWebOfLies133CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT359 |
| ✓ | BbhWebOfLies134CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT321 |
| ✓ | BbhWebOfLies135CRtruth-teller/liar chain — Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT379 |
| ✓ | BbhWebOfLies136CRtruth-teller/liar chain — Does Conception tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT289 |
| ✓ | BbhWebOfLies137CRtruth-teller/liar chain — Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT242 |
| ✓ | BbhWebOfLies138CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT343 |
Reasoning - BBH web of lies (set 5)
32 passed |
100.0% |
IN32 010 OUT9 332 | |
| ✓ | BbhWebOfLies139CRtruth-teller/liar chain — Does Fletcher tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT226 |
| ✓ | BbhWebOfLies140CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT248 |
| ✓ | BbhWebOfLies141CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT324 |
| ✓ | BbhWebOfLies142CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT310 |
| ✓ | BbhWebOfLies143CRtruth-teller/liar chain — Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT329 |
| ✓ | BbhWebOfLies144CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT334 |
| ✓ | BbhWebOfLies145CRtruth-teller/liar chain — Does Fidel tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT226 |
| ✓ | BbhWebOfLies146CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT304 |
| ✓ | BbhWebOfLies147CRtruth-teller/liar chain — Does Millie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT294 |
| ✓ | BbhWebOfLies148CRtruth-teller/liar chain — Does Audrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT280 |
| ✓ | BbhWebOfLies149CRtruth-teller/liar chain — Does Phoebe tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT320 |
| ✓ | BbhWebOfLies150CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT265 |
| ✓ | BbhWebOfLies151CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT240 |
| ✓ | BbhWebOfLies152CRtruth-teller/liar chain — Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT330 |
| ✓ | BbhWebOfLies153CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT294 |
| ✓ | BbhWebOfLies154CRtruth-teller/liar chain — Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT225 |
| ✓ | BbhWebOfLies155CRtruth-teller/liar chain — Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT259 |
| ✓ | BbhWebOfLies156CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT255 |
| ✓ | BbhWebOfLies157CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT235 |
| ✓ | BbhWebOfLies158CRtruth-teller/liar chain — Does Shalonda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT307 |
| ✓ | BbhWebOfLies159CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT318 |
| ✓ | BbhWebOfLies160CRtruth-teller/liar chain — Does Ryan tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT324 |
| ✓ | BbhWebOfLies161CRtruth-teller/liar chain — Does Willian tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT269 |
| ✓ | BbhWebOfLies162CRtruth-teller/liar chain — Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT290 |
| ✓ | BbhWebOfLies163CRtruth-teller/liar chain — Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT228 |
| ✓ | BbhWebOfLies164CRtruth-teller/liar chain — Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT228 |
| ✓ | BbhWebOfLies165CRtruth-teller/liar chain — Does Raymond tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT320 |
| ✓ | BbhWebOfLies166CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 010 OUT337 |
| ✓ | BbhWebOfLies167CRtruth-teller/liar chain — Does Rashida tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT329 |
| ✓ | BbhWebOfLies168CRtruth-teller/liar chain — Does Leda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT312 |
| ✓ | BbhWebOfLies169CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT432 |
| ✓ | BbhWebOfLies170CRtruth-teller/liar chain — Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT340 |
Reasoning - BBH web of lies (set 6)
30 passed |
100.0% |
IN29 986 OUT9 315 | |
| ✓ | BbhWebOfLies171CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT289 |
| ✓ | BbhWebOfLies172CRtruth-teller/liar chain — Does Bernita tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT364 |
| ✓ | BbhWebOfLies173CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT343 |
| ✓ | BbhWebOfLies174CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT288 |
| ✓ | BbhWebOfLies175CRtruth-teller/liar chain — Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT320 |
| ✓ | BbhWebOfLies176CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT337 |
| ✓ | BbhWebOfLies177CRtruth-teller/liar chain — Does Phoebe tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT278 |
| ✓ | BbhWebOfLies178CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT294 |
| ✓ | BbhWebOfLies179CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT329 |
| ✓ | BbhWebOfLies180CRtruth-teller/liar chain — Does Sherrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT264 |
| ✓ | BbhWebOfLies181CRtruth-teller/liar chain — Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT298 |
| ✓ | BbhWebOfLies182CRtruth-teller/liar chain — Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT334 |
| ✓ | BbhWebOfLies183CRtruth-teller/liar chain — Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT312 |
| ✓ | BbhWebOfLies184CRtruth-teller/liar chain — Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT329 |
| ✓ | BbhWebOfLies185CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT355 |
| ✓ | BbhWebOfLies186CRtruth-teller/liar chain — Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT322 |
| ✓ | BbhWebOfLies187CRtruth-teller/liar chain — Does Fletcher tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT296 |
| ✓ | BbhWebOfLies188CRtruth-teller/liar chain — Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT246 |
| ✓ | BbhWebOfLies189CRtruth-teller/liar chain — Does Helene tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT310 |
| ✓ | BbhWebOfLies190CRtruth-teller/liar chain — Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT308 |
| ✓ | BbhWebOfLies191CRtruth-teller/liar chain — Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT323 |
| ✓ | BbhWebOfLies192CRtruth-teller/liar chain — Does Teressa tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT306 |
| ✓ | BbhWebOfLies193CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT267 |
| ✓ | BbhWebOfLies194CRtruth-teller/liar chain — Does Dallas tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT411 |
| ✓ | BbhWebOfLies195CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT248 |
| ✓ | BbhWebOfLies196CRtruth-teller/liar chain — Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT261 |
| ✓ | BbhWebOfLies197CRtruth-teller/liar chain — Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT239 |
| ✓ | BbhWebOfLies198CRtruth-teller/liar chain — Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT392 |
| ✓ | BbhWebOfLies199CRtruth-teller/liar chain — Does Kandi tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT306 |
| ✓ | BbhWebOfLies200CRtruth-teller/liar chain — Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT346 |
Reasoning - BBH web of lies (set 7)
32 passed |
100.0% |
IN32 002 OUT10 696 | |
| ✓ | BbhWebOfLies201CSJamey tells the truth. Michael says Jamey lies. Sherrie says Michael lies. Bernita says Sherrie lies. Amberly says Bernita tells the truth. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT311 |
| ✓ | BbhWebOfLies202CSShaunda tells the truth. Jim says Shaunda lies. Alexis says Jim lies. Phoebe says Alexis tells the truth. Sherrie says Phoebe tells the truth. Does Sherrie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT548 |
| ✓ | BbhWebOfLies203CSMillicent lies. Ka says Millicent lies. Sima says Ka tells the truth. Dallas says Sima tells the truth. Jamey says Dallas lies. Does Jamey tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT280 |
| ✓ | BbhWebOfLies204CSChristie tells the truth. Fidel says Christie lies. Jerry says Fidel tells the truth. Maybelle says Jerry lies. Delfina says Maybelle tells the truth. Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT322 |
| ✓ | BbhWebOfLies205CSSima lies. Maybelle says Sima tells the truth. Audrie says Maybelle lies. Leda says Audrie tells the truth. Shaunda says Leda lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT314 |
| ✓ | BbhWebOfLies206CSVernell lies. Conception says Vernell lies. Kandi says Conception lies. Michael says Kandi tells the truth. Osvaldo says Michael tells the truth. Does Osvaldo tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT350 |
| ✓ | BbhWebOfLies207CSShaunda tells the truth. Delbert says Shaunda tells the truth. Bernita says Delbert lies. Yoland says Bernita tells the truth. Christie says Yoland tells the truth. Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT237 |
| ✓ | BbhWebOfLies208CSInga tells the truth. Shalonda says Inga tells the truth. Phoebe says Shalonda tells the truth. Crista says Phoebe lies. Alejandro says Crista lies. Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT243 |
| ✓ | BbhWebOfLies209CSFidel lies. Millicent says Fidel tells the truth. Crista says Millicent lies. Leda says Crista lies. Tamika says Leda tells the truth. Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT284 |
| ✓ | BbhWebOfLies210CSAndree tells the truth. Christie says Andree tells the truth. Fidel says Christie lies. Antwan says Fidel tells the truth. Alejandro says Antwan lies. Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT258 |
| ✓ | BbhWebOfLies211CSFletcher tells the truth. Willian says Fletcher tells the truth. Jamey says Willian lies. Leda says Jamey lies. Jaymie says Leda tells the truth. Does Jaymie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT321 |
| ✓ | BbhWebOfLies212CSTamika lies. Jerry says Tamika tells the truth. Delbert says Jerry tells the truth. Leda says Delbert tells the truth. Willian says Leda lies. Does Willian tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT439 |
| ✓ | BbhWebOfLies213CSTeressa lies. Sal says Teressa tells the truth. Delbert says Sal tells the truth. Tamika says Delbert tells the truth. Gwenn says Tamika tells the truth. Does Gwenn tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT371 |
| ✓ | BbhWebOfLies214CSJim lies. Delfina says Jim tells the truth. Shaunda says Delfina lies. Fidel says Shaunda lies. Sal says Fidel tells the truth. Does Sal tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT361 |
| ✓ | BbhWebOfLies215CSMillie lies. Michael says Millie tells the truth. Vina says Michael lies. Conception says Vina tells the truth. Sima says Conception tells the truth. Does Sima tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT357 |
| ✓ | BbhWebOfLies216CSKandi lies. Millie says Kandi tells the truth. Lorine says Millie tells the truth. Michaela says Lorine tells the truth. Amberly says Michaela lies. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT334 |
| ✓ | BbhWebOfLies217CSMichael lies. Jerry says Michael tells the truth. Delbert says Jerry tells the truth. Sima says Delbert lies. Alexis says Sima lies. Does Alexis tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT322 |
| ✓ | BbhWebOfLies218CSSal lies. Kristian says Sal lies. Helene says Kristian lies. Inga says Helene tells the truth. Michael says Inga tells the truth. Does Michael tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT322 |
| ✓ | BbhWebOfLies219CSMaybelle lies. Crista says Maybelle tells the truth. Bernita says Crista lies. Kandi says Bernita tells the truth. Delbert says Kandi tells the truth. Does Delbert tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT381 |
| ✓ | BbhWebOfLies220CSKa lies. Rashida says Ka tells the truth. Sherrie says Rashida tells the truth. Tamika says Sherrie tells the truth. Elanor says Tamika tells the truth. Does Elanor tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT339 |
| ✓ | BbhWebOfLies221CSAntwan tells the truth. Audrie says Antwan tells the truth. Inga says Audrie lies. Shenna says Inga tells the truth. Christie says Shenna lies. Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT232 |
| ✓ | BbhWebOfLies222CSTeressa lies. Michael says Teressa tells the truth. Fidel says Michael tells the truth. Leda says Fidel lies. Lorine says Leda lies. Does Lorine tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT345 |
| ✓ | BbhWebOfLies223CSAntwan tells the truth. Elanor says Antwan tells the truth. Bernita says Elanor tells the truth. Osvaldo says Bernita lies. Christie says Osvaldo tells the truth. Does Christie tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT311 |
| ✓ | BbhWebOfLies224CSInga tells the truth. Elanor says Inga lies. Millicent says Elanor tells the truth. Willian says Millicent tells the truth. Alejandro says Willian tells the truth. Does Alejandro tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 006 OUT338 |
| ✓ | BbhWebOfLies225CSSherrie lies. Elanor says Sherrie tells the truth. Michael says Elanor tells the truth. Sima says Michael tells the truth. Shaunda says Sima lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT322 |
| ✓ | BbhWebOfLies226CSTamika lies. Conception says Tamika tells the truth. Kristian says Conception lies. Alejandro says Kristian tells the truth. Shaunda says Alejandro lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT361 |
| ✓ | BbhWebOfLies227CSHelene lies. Delfina says Helene lies. Christie says Delfina tells the truth. Jerry says Christie lies. Tamika says Jerry tells the truth. Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT277 |
| ✓ | BbhWebOfLies228CSConception lies. Shenna says Conception tells the truth. Maybelle says Shenna tells the truth. Michael says Maybelle lies. Amberly says Michael lies. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT545 |
| ✓ | BbhWebOfLies229CSKristian tells the truth. Jamey says Kristian tells the truth. Millicent says Jamey lies. Fidel says Millicent lies. Tamika says Fidel lies. Does Tamika tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT272 |
| ✓ | BbhWebOfLies230CSJamey lies. Sal says Jamey tells the truth. Teressa says Sal lies. Maybelle says Teressa tells the truth. Ka says Maybelle lies. Does Ka tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT295 |
| ✓ | BbhWebOfLies231CSLorine lies. Amberly says Lorine lies. Fletcher says Amberly lies. Millie says Fletcher tells the truth. Antwan says Millie lies. Does Antwan tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT371 |
| ✓ | BbhWebOfLies232CSAlexis tells the truth. Elanor says Alexis tells the truth. Sherrie says Elanor lies. Audrie says Sherrie tells the truth. Inga says Audrie tells the truth. Does Inga tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT333 |
Reasoning - BBH web of lies (set 8)
31 passed1 failed |
96.9% |
IN31 729 OUT13 907 | |
| ✓ | BbhWebOfLies233CSFidel tells the truth. Teressa says Fidel tells the truth. Willian says Teressa tells the truth. Conception says Willian tells the truth. Bernita says Conception lies. Does Bernita tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT307 |
| ✓ | BbhWebOfLies234CSJerry tells the truth. Lorine says Jerry lies. Shenna says Lorine lies. Gwenn says Shenna lies. Shaunda says Gwenn lies. Does Shaunda tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT288 |
| ✓ | BbhWebOfLies235CSInga lies. Audrie says Inga lies. Rashida says Audrie lies. Willian says Rashida tells the truth. Amberly says Willian lies. Does Amberly tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT344 |
| ✓ | BbhWebOfLies236CSKa tells the truth. Inga says Ka lies. Bernita says Inga lies. Osvaldo says Bernita tells the truth. Michaela says Osvaldo lies. Does Michaela tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT341 |
| ✓ | BbhWebOfLies237CSJaymie lies. Jamey says Jaymie tells the truth. Sal says Jamey lies. Amberly says Sal lies. Jerry says Amberly tells the truth. Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT339 |
| ✓ | BbhWebOfLies238CSMillie lies. Andree says Millie lies. Michaela says Andree lies. Leda says Michaela tells the truth. Shenna says Leda lies. Does Shenna tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN998 OUT278 |
| ✓ | BbhWebOfLies239CSShaunda tells the truth. Maybelle says Shaunda tells the truth. Lorine says Maybelle lies. Millie says Lorine tells the truth. Vina says Millie tells the truth. Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT346 |
| ✓ | BbhWebOfLies240CSFletcher lies. Tamika says Fletcher tells the truth. Ka says Tamika lies. Sima says Ka tells the truth. Ryan says Sima tells the truth. Does Ryan tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT283 |
| ✓ | BbhWebOfLies241CSShaunda lies. Andree says Shaunda tells the truth. Jaymie says Andree lies. Inga says Jaymie lies. Millicent says Inga tells the truth. Does Millicent tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT289 |
| ✓ | BbhWebOfLies242CSRashida lies. Fletcher says Rashida lies. Antwan says Fletcher lies. Willian says Antwan tells the truth. Vina says Willian tells the truth. Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT362 |
| ✓ | BbhWebOfLies243CSJerry lies. Kandi says Jerry lies. Fletcher says Kandi lies. Ryan says Fletcher lies. Delfina says Ryan lies. Does Delfina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN992 OUT318 |
| ✓ | BbhWebOfLies244CSShaunda tells the truth. Phoebe says Shaunda tells the truth. Ryan says Phoebe tells the truth. Christie says Ryan tells the truth. Vina says Christie tells the truth. Does Vina tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT215 |
| ✓ | BbhWebOfLies245CSAlexis tells the truth. Amberly says Alexis tells the truth. Andree says Amberly tells the truth. Vina says Andree tells the truth. Helene says Vina tells the truth. Does Helene tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 002 OUT257 |
| ✓ | BbhWebOfLies246CSGwenn lies. Maybelle says Gwenn tells the truth. Raymond says Maybelle tells the truth. Jamey says Raymond tells the truth. Andree says Jamey lies. Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 004 OUT351 |
| ✓ | BbhWebOfLies247CSKandi tells the truth. Ryan says Kandi lies. Michaela says Ryan tells the truth. Sima says Michaela lies. Andree says Sima tells the truth. Does Andree tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN1 000 OUT312 |
| ✓ | BbhWebOfLies248CSFletcher tells the truth. Alexis says Fletcher lies. Jerry says Alexis lies. Elanor says Jerry tells the truth. Jim says Elanor lies. Does Jim tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN994 OUT294 |
| ✓ | BbhWebOfLies249CSSherrie tells the truth. Ryan says Sherrie lies. Bernita says Ryan tells the truth. Tamika says Bernita lies. Jerry says Tamika lies. Does Jerry tell the truth?1 AI query
0 tool callscategory:Reasoninglevel:Hardsource:BBH |
Passed | IN996 OUT237 |
| ✓ | BbhWordSorting10CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT403 |
| ✓ | BbhWordSorting11CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 002 OUT994 |
| ✓ | BbhWordSorting12CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT771 |
| ✓ | BbhWordSorting13CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT647 |
| ✓ | BbhWordSorting14CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT306 |
| ✓ | BbhWordSorting15CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT1 284 |
| × | BbhWordSorting16CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"advent anger convoy deliver filly gneiss grocer hessian hotbox landau marlborough ninebark plat platelet pyrotechnic siemens stapleton transitive treadle uncle")
Context: BbhWordSorting16CS: expected advent anger convoy deliver filly gneiss grocer hessian hotbox landau marlborough ninebark plat platelet pyrotechnic siemens stapleton transitive treadle uncle, got [advent anger convoy deliver filly gneiss grocer hessian hotbox landau marlborough ninebark plat platelet pyrotechnic siemens stapleton treadle transitive uncle] |
Failed | IN1 003 OUT989 |
| ✓ | BbhWordSorting17CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT641 |
| ✓ | BbhWordSorting18CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 121 |
| ✓ | BbhWordSorting19CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT222 |
| ✓ | BbhWordSorting20CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT134 |
| ✓ | BbhWordSorting21CSalphabetically sort 7 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT314 |
| ✓ | BbhWordSorting22CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT518 |
| ✓ | BbhWordSorting8CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT125 |
| ✓ | BbhWordSorting9CSalphabetically sort 6 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT277 |
Reasoning - BBH word sorting (set 1)
28 passed4 failed |
87.5% |
IN31 451 OUT17 050 | |
| ✓ | BbhWordSorting23CSalphabetically sort 7 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT338 |
| × | BbhWordSorting24CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"confess croupier daffy dockyard duty household hypothesis info loam mandate mantic minstrelsy nepotism peccary sawtimber serenade silver summate triode")
Context: BbhWordSorting24CS: expected confess croupier daffy dockyard duty household hypothesis info loam mandate mantic minstrelsy nepotism peccary sawtimber serenade silver summate triode, got [confess croupier daffy dockyard duty household hypothesis info loam mandate mantic minstrelsy nepotism peccary serenade silver summate triode] |
Failed | IN1 001 OUT1 019 |
| ✓ | BbhWordSorting25CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 055 |
| ✓ | BbhWordSorting26CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT706 |
| ✓ | BbhWordSorting27CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT198 |
| ✓ | BbhWordSorting28CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT640 |
| ✓ | BbhWordSorting29CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT664 |
| ✓ | BbhWordSorting30CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT730 |
| ✓ | BbhWordSorting31CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT451 |
| ✓ | BbhWordSorting32CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT120 |
| × | BbhWordSorting33CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"adonis birdseed citizen contaminant convair extensive fateful frighten judaica scrubby soothe southeastern stormy suppose trillion trundle")
Context: BbhWordSorting33CS: expected adonis birdseed citizen contaminant convair extensive fateful frighten judaica scrubby soothe southeastern stormy suppose trillion trundle, got [adonis birdseed citizen contaminant convair extensive fateful frighten judaica scrubby soothe southeastern stormy suppose trundle trillion] |
Failed | IN992 OUT948 |
| ✓ | BbhWordSorting34CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT175 |
| × | BbhWordSorting35CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"belize bolshevism cost dance deadline dietetic formulae foster hesitant huddle judson mantle odessa palace progeny proust rackety resplendent thirdhand warmth")
Context: BbhWordSorting35CS: expected belize bolshevism cost dance deadline dietetic formulae foster hesitant huddle judson mantle odessa palace progeny proust rackety resplendent thirdhand warmth, got [belize bolshevism cost dance deadline dietetic foster hesitant huddle judson mantle odessa palace progeny proust rackety resplendent thirdhand warmth] |
Failed | IN998 OUT1 114 |
| ✓ | BbhWordSorting36CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT387 |
| ✓ | BbhWordSorting37CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 183 |
| ✓ | BbhWordSorting38CSalphabetically sort 2 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT101 |
| ✓ | BbhWordSorting39CSalphabetically sort 2 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT105 |
| ✓ | BbhWordSorting40CSalphabetically sort 14 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT761 |
| × | BbhWordSorting41CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"arraign blutwurst convenient faber glacier horizon inconspicuous peste portentous rancho uranyl")
Context: BbhWordSorting41CS: expected arraign blutwurst convenient faber glacier horizon inconspicuous peste portentous rancho uranyl, got [arrain blutwurst convenient faber glacier horizon inconspicuous peste portentous rancho uranyl] |
Failed | IN986 OUT552 |
| ✓ | BbhWordSorting42CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT120 |
| ✓ | BbhWordSorting43CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT437 |
| ✓ | BbhWordSorting44CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 001 OUT1 154 |
| ✓ | BbhWordSorting45CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT509 |
| ✓ | BbhWordSorting46CSalphabetically sort 6 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT451 |
| ✓ | BbhWordSorting47CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT351 |
| ✓ | BbhWordSorting48CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT367 |
| ✓ | BbhWordSorting49CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT907 |
| ✓ | BbhWordSorting50CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT117 |
| ✓ | BbhWordSorting51CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT448 |
| ✓ | BbhWordSorting52CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT295 |
| ✓ | BbhWordSorting53CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT164 |
| ✓ | BbhWordSorting54CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT483 |
Reasoning - BBH word sorting (set 2)
27 passed5 failed |
84.4% |
IN31 667 OUT24 983 | |
| ✓ | BbhWordSorting55CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT775 |
| ✓ | BbhWordSorting56CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT362 |
| ✓ | BbhWordSorting57CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 012 OUT1 004 |
| × | BbhWordSorting58CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"auxin awash bateau cubit eutectic gown gullible inane jurisprudential mistletoe nepenthe ow pirouette pussycat schwartz scottsdale shockley travelogue upbring")
Context: BbhWordSorting58CS: expected auxin awash bateau cubit eutectic gown gullible inane jurisprudential mistletoe nepenthe ow pirouette pussycat schwartz scottsdale shockley travelogue upbring, got [awash auxin bateau cubit eutectic gown gullible inane jurisprudential mistletoe nepenthe ow pirouette pussycat scottsdale shockley schwartz travelogue upbring] |
Failed | IN1 007 OUT935 |
| ✓ | BbhWordSorting59CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT1 076 |
| ✓ | BbhWordSorting60CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT645 |
| ✓ | BbhWordSorting61CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT721 |
| ✓ | BbhWordSorting62CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 003 OUT1 196 |
| ✓ | BbhWordSorting63CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT541 |
| ✓ | BbhWordSorting64CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT445 |
| ✓ | BbhWordSorting65CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT841 |
| ✓ | BbhWordSorting66CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT757 |
| × | BbhWordSorting67CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"allele anthropocentric badinage banish bartok brunswick dale dar desolater dun fraternity goat martinson monomer morphemic pegging starkey underclassmen whoop yourselves")
Context: BbhWordSorting67CS: expected allele anthropocentric badinage banish bartok brunswick dale dar desolater dun fraternity goat martinson monomer morphemic pegging starkey underclassmen whoop yourselves, got [allele anthropocentric badinage banish bartok brunswick dale dar desolater dun fraternity goat martinson monomer morphemic pegging selves starkey underclassmen whoop yours] |
Failed | IN1 001 OUT2 234 |
| ✓ | BbhWordSorting68CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT318 |
| ✓ | BbhWordSorting69CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT540 |
| × | BbhWordSorting70CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"booby butadiene flair functor heck orphanage racy rheumatic shivery sin snowball spec testy trench zorn")
Context: BbhWordSorting70CS: expected booby butadiene flair functor heck orphanage racy rheumatic shivery sin snowball spec testy trench zorn, got [booby butadiene flair functor heck orphanage racy rheumatic shivery sin snowball spec trench testy zorn] |
Failed | IN989 OUT596 |
| ✓ | BbhWordSorting71CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT936 |
| ✓ | BbhWordSorting72CSalphabetically sort 8 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT385 |
| ✓ | BbhWordSorting73CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT549 |
| ✓ | BbhWordSorting74CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 003 OUT1 370 |
| ✓ | BbhWordSorting75CSalphabetically sort 5 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT177 |
| × | BbhWordSorting76CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"astigmat boyish coriolanus creak cutlet easternmost godson heaven highwaymen leather muscular musky paula scavenge synaptic zinc")
Context: BbhWordSorting76CS: expected astigmat boyish coriolanus creak cutlet easternmost godson heaven highwaymen leather muscular musky paula scavenge synaptic zinc, got [boyish coriolanus creak cutlet easternmost godson heaven highwaymen leather muscular musky paula scavenge synaptic zinc] |
Failed | IN993 OUT660 |
| ✓ | BbhWordSorting77CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT132 |
| ✓ | BbhWordSorting78CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT746 |
| ✓ | BbhWordSorting79CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT820 |
| ✓ | BbhWordSorting80CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 008 OUT1 576 |
| ✓ | BbhWordSorting81CSalphabetically sort 20 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 259 |
| ✓ | BbhWordSorting82CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT155 |
| ✓ | BbhWordSorting83CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT602 |
| ✓ | BbhWordSorting84CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT296 |
| ✓ | BbhWordSorting85CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT683 |
| × | BbhWordSorting86CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"betony boar bootleg bronzy centaur charge clemens collet contemporaneous contravariant cordial dorado handicraft macmahon mesh monterey possession regina underclassman")
Context: BbhWordSorting86CS: expected betony boar bootleg bronzy centaur charge clemens collet contemporaneous contravariant cordial dorado handicraft macmahon mesh monterey possession regina underclassman, got [betony boar bootleg bronzy centaur charge clemens contemporaneous contravariant cordial dorado handicraft macmahon mesh monterey possession regina underclassman] |
Failed | IN1 002 OUT1 651 |
Reasoning - BBH word sorting (set 3)
28 passed2 failed |
93.3% |
IN29 651 OUT22 104 | |
| ✓ | BbhWordSorting100CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT157 |
| ✓ | BbhWordSorting101CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 007 |
| × | BbhWordSorting102CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"adsorption align anastasia anastomotic apache award bobbin burrow calumny epaulet execrable hostelry hun macedon omnipotent putty roughshod smooth spontaneity")
Context: BbhWordSorting102CS: expected adsorption align anastasia anastomotic apache award bobbin burrow calumny epaulet execrable hostelry hun macedon omnipotent putty roughshod smooth spontaneity, got [align anastasia anastomotic apache award bobbin burrow calumny epaulet execrable hostelry hun macedon omnipotent putty roughshod smooth spontaneity] |
Failed | IN1 006 OUT1 760 |
| ✓ | BbhWordSorting103CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT378 |
| ✓ | BbhWordSorting104CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT557 |
| ✓ | BbhWordSorting105CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT787 |
| ✓ | BbhWordSorting106CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 024 |
| ✓ | BbhWordSorting107CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT916 |
| ✓ | BbhWordSorting108CSalphabetically sort 11 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT516 |
| ✓ | BbhWordSorting109CSalphabetically sort 8 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT564 |
| ✓ | BbhWordSorting110CSalphabetically sort 9 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT465 |
| ✓ | BbhWordSorting111CSalphabetically sort 14 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT595 |
| ✓ | BbhWordSorting112CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN990 OUT685 |
| ✓ | BbhWordSorting113CSalphabetically sort 10 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT766 |
| ✓ | BbhWordSorting114CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT590 |
| ✓ | BbhWordSorting115CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT710 |
| × | BbhWordSorting116CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm(@"christen clearheaded despond driveway encapsulate fungi gob mendelevium midwinter purpose sisyphus stanhope strip studious symmetry trample vs wring")
Context: BbhWordSorting116CS: expected christen clearheaded despond driveway encapsulate fungi gob mendelevium midwinter purpose sisyphus stanhope strip studious symmetry trample vs wring, got [christen clearheaded driveway despond encapsulate fungi gob midwinter mendelevium purpose sisyphus stanhope strip studious symmetry trample vs wring] |
Failed | IN998 OUT1 730 |
| ✓ | BbhWordSorting87CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT617 |
| ✓ | BbhWordSorting88CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT125 |
| ✓ | BbhWordSorting89CSalphabetically sort 18 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT1 073 |
| ✓ | BbhWordSorting90CSalphabetically sort 19 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 657 |
| ✓ | BbhWordSorting91CSalphabetically sort 17 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT793 |
| ✓ | BbhWordSorting92CSalphabetically sort 4 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT172 |
| ✓ | BbhWordSorting93CSalphabetically sort 16 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT1 338 |
| ✓ | BbhWordSorting94CSalphabetically sort 13 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT535 |
| ✓ | BbhWordSorting95CSalphabetically sort 12 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT657 |
| ✓ | BbhWordSorting96CSalphabetically sort 15 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT780 |
| ✓ | BbhWordSorting97CSalphabetically sort 14 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT921 |
| ✓ | BbhWordSorting98CSalphabetically sort 2 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN966 OUT100 |
| ✓ | BbhWordSorting99CSalphabetically sort 3 words1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT129 |
Reasoning - BBH word sorting (set 4)
26 passed1 failed |
96.3% |
IN26 570 OUT16 061 | |
| ✓ | BbhWordSorting117CTSort the following words alphabetically: List: marshmallow doge alpenstock ocean accept angus drool jutish resistive chromium vociferous castigate prize octennial stonewort concision elizabethan1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT1 011 |
| ✓ | BbhWordSorting118CTSort the following words alphabetically: List: jocund flagellate bodyguard flotation commensal involve miff ineradicable postprocess1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT504 |
| ✓ | BbhWordSorting119CTSort the following words alphabetically: List: gloriana milord rendezvous schnabel allegoric loge lollipop salle collate prolix mast euphony1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT665 |
| ✓ | BbhWordSorting120CTSort the following words alphabetically: List: pea yonkers darkle quadrant moldboard instant olsen hookup erudite lip1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT298 |
| ✓ | BbhWordSorting121CTSort the following words alphabetically: List: regret uptake satisfy behold fumarole motif forth alcohol escutcheon pease hackberry walkie1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT886 |
| ✓ | BbhWordSorting122CTSort the following words alphabetically: List: charcuterie muddle diatom impenetrable footfall crucifix greenberg synchronous spoken1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT760 |
| ✓ | BbhWordSorting123CTSort the following words alphabetically: List: sibyl calligraph goat inverness form threadbare1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT254 |
| ✓ | BbhWordSorting124CTSort the following words alphabetically: List: cashew apprehension ensemble1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT116 |
| ✓ | BbhWordSorting125CTSort the following words alphabetically: List: seashore haughty1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT111 |
| ✓ | BbhWordSorting126CTSort the following words alphabetically: List: postcondition protoplasmic musicology helical uptrend vasoconstriction diacritic beefsteak beware birthplace bicycle junctor state obstinate banshee sap1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT1 195 |
| ✓ | BbhWordSorting127CTSort the following words alphabetically: List: whisper eyesight titmice mirage fletch cry biennial creedal miltonic fraudulent j1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT623 |
| ✓ | BbhWordSorting128CTSort the following words alphabetically: List: multitudinous adopt wage worcestershire pacifist afghan glimmer friday1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT661 |
| ✓ | BbhWordSorting129CTSort the following words alphabetically: List: lethe bilk perturb tactual1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT165 |
| ✓ | BbhWordSorting130CTSort the following words alphabetically: List: liggett sterno pintail dilate frolic piggish forth parameter slaughterhouse lacunae mcintosh bloodstain lazy unesco protector beachhead algonquin1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 440 |
| ✓ | BbhWordSorting131CTSort the following words alphabetically: List: erudite marriageable stahl coach inadvisable vicksburg virgo brownian eosine magnesium flax1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT581 |
| ✓ | BbhWordSorting132CTSort the following words alphabetically: List: clifford brindle siskin gloat florist triploidy willard sacramento1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT438 |
| ✓ | BbhWordSorting133CTSort the following words alphabetically: List: confrontation quantitative proserpine proofread daddy hirsute1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT534 |
| ✓ | BbhWordSorting134CTSort the following words alphabetically: List: allyn carbonaceous majorca investigatory supersede cetacean pathogenic johann tung paradigmatic pray1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN985 OUT525 |
| ✓ | BbhWordSorting135CTSort the following words alphabetically: List: calyx emphases vii elgin perquisite implacable blackbird afro jura mayapple whit blame1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT509 |
| ✓ | BbhWordSorting136CTSort the following words alphabetically: List: cast palazzi inhibit arragon tabletop danbury abbe adposition emplace gavin income falsetto onerous1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT573 |
| × | BbhWordSorting137CTSort the following words alphabetically: List: ulcerate ballroom sparse intrepid arcing prorogue dodge benefit coupon ladle buggy shagging bassoon hypothermia decide affirmative airframe junior nineveh schmitt1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("affirmative airframe arcing ballroom bassoon benefit buggy coupon decide dodge hypothermia intrepid junior ladle nineveh prorogue schmitt shagging sparse ulcerate")
Context: BbhWordSorting137CT: expected sorted list, got [affirmative airframe arcing ballroom bassoon benefit buggy decide dodge hypothermia intrepid junior ladle nineveh prorogue schmitt shagging sparse ulcerate] |
Failed | IN998 OUT834 |
| ✓ | BbhWordSorting138CTSort the following words alphabetically: List: statuette whiteface lauderdale encumber atonic specify olivia adipic neutrino antique catch persona athlete sovereignty1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT632 |
| ✓ | BbhWordSorting139CTSort the following words alphabetically: List: hyperboloidal borough1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT112 |
| ✓ | BbhWordSorting140CTSort the following words alphabetically: List: yves butt scribe seaworthy manumitted clapeyron lift blackguard reconnaissance agile woodcut lightfooted sawbelly meager geophysical wiseacre mathieu cognoscenti purposive flamboyant1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 007 OUT1 013 |
| ✓ | BbhWordSorting141CTSort the following words alphabetically: List: duane harriet impressible whippet glamour cowbell vietnamese fest switchboard anaglyph texture1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT443 |
| ✓ | BbhWordSorting142CTSort the following words alphabetically: List: plural dose allstate stalin dyad multitudinous powderpuff1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT499 |
| ✓ | BbhWordSorting143CTSort the following words alphabetically: List: crossbill bushland capacitive meager ricochet battery enigma wysiwyg wallet lipton jane wacke contingent1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT679 |
Reasoning - BBH word sorting (set 5)
24 passed3 failed |
88.9% |
IN26 547 OUT15 716 | |
| × | BbhWordSorting144CTSort the following words alphabetically: List: corpulent diagnose code scalp bombproof fluorine blythe swipe honeybee damn pore maharaja cytolysis solicit1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("blythe bombproof code corpulent cytolysis damn diagnose fluorine honeybee maharaja pore scalp solicit swipe")
Context: BbhWordSorting144CT: expected sorted list, got [bombproof blythe code corpulent cytolysis damn diagnose fluorine honeybee maharaja pore scalp solicit swipe] |
Failed | IN988 OUT949 |
| ✓ | BbhWordSorting145CTSort the following words alphabetically: List: fugue chicanery mountain1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT123 |
| ✓ | BbhWordSorting146CTSort the following words alphabetically: List: sideboard procrustean seepage buxton olympic tty typescript marque callus unitary cameron contribute precise verify shelf extensible methanol1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT966 |
| ✓ | BbhWordSorting147CTSort the following words alphabetically: List: sociolinguistic nigeria odysseus demystify proton voltaire penny contaminate bighorn stirrup1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN987 OUT393 |
| ✓ | BbhWordSorting148CTSort the following words alphabetically: List: greasy lapidary mark1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT121 |
| ✓ | BbhWordSorting149CTSort the following words alphabetically: List: choreograph retardant multifarious pro carbonic equestrian wishful wigwam countersink libya authenticate metal have corvallis nitric petition obfuscatory1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT911 |
| ✓ | BbhWordSorting150CTSort the following words alphabetically: List: oakland seaborg jacobi membrane trapezoidal allis marmot toggle anthology1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT714 |
| ✓ | BbhWordSorting151CTSort the following words alphabetically: List: monotonous slick provocation functor hideaway holeable assimilable lola calypso nebuchadnezzar dogwood pacifism bivariate bought1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT639 |
| ✓ | BbhWordSorting152CTSort the following words alphabetically: List: livre substitution ix frisian panoramic buckley1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN972 OUT267 |
| ✓ | BbhWordSorting153CTSort the following words alphabetically: List: no spokane cartilaginous that'd science1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT344 |
| ✓ | BbhWordSorting154CTSort the following words alphabetically: List: splashy rectitude novelty1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT118 |
| ✓ | BbhWordSorting155CTSort the following words alphabetically: List: sediment varsity towel fracture1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT164 |
| ✓ | BbhWordSorting156CTSort the following words alphabetically: List: confidential ursa fiction heterozygous faery overture horehound1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT432 |
| ✓ | BbhWordSorting157CTSort the following words alphabetically: List: mckinley pesticide school radiometer marque prig relish trianon inspiration artful cancelled castrate glaucous ear croon tioga rothschild excite endpoint citadel1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT1 944 |
| ✓ | BbhWordSorting158CTSort the following words alphabetically: List: iniquity yoghurt behind volcanism hornpipe mcconnell scorn zip toroidal mollie sandy wellwisher inmate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT792 |
| ✓ | BbhWordSorting159CTSort the following words alphabetically: List: extempore gotten1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT109 |
| ✓ | BbhWordSorting160CTSort the following words alphabetically: List: yang relinquish merry maladjust olefin canaan ranch batavia1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN977 OUT402 |
| ✓ | BbhWordSorting161CTSort the following words alphabetically: List: yuh edt cheddar oblivion from poignant pang1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT321 |
| ✓ | BbhWordSorting162CTSort the following words alphabetically: List: zan natalie coplanar stevenson1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT142 |
| ✓ | BbhWordSorting163CTSort the following words alphabetically: List: bottle kenney chapati okinawa boon bedtime1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT541 |
| × | BbhWordSorting164CTSort the following words alphabetically: List: contrariety hoop muong dimorphic crochet forthwith hurty titrate prodigy polytope guaranteeing atavism emanate iniquity grind more katie1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("atavism contrariety crochet dimorphic emanate forthwith grind guaranteeing hoop hurty iniquity katie more muong polytope prodigy titrate")
Context: BbhWordSorting164CT: expected sorted list, got [atavism crochet contrariety dimorphic emanate forthwith grind guaranteeing hoop hurty iniquity katie more muong polytope prodigy titrate] |
Failed | IN998 OUT771 |
| ✓ | BbhWordSorting165CTSort the following words alphabetically: List: sloane molten equip hillman max cochran practise advisee kuwait chancery hoofprint cardigan evildoer retinue wuhan ape apostate acrobacy crowbait1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 002 OUT841 |
| ✓ | BbhWordSorting166CTSort the following words alphabetically: List: catawba almagest transfusable milt photolysis limpid gnome archenemy sagging councilwomen decrement jungian1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT574 |
| × | BbhWordSorting167CTSort the following words alphabetically: List: inscribe upraise perceptive tragedian peculate cunard posterior crude danubian1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("crude cunard danubian inscribe peculate perceptive posterior tragedian upraise")
Context: BbhWordSorting167CT: expected sorted list, got [cunard crude danubian inscribe peculate perceptive posterior tragedian upraise] |
Failed | IN982 OUT724 |
| ✓ | BbhWordSorting168CTSort the following words alphabetically: List: madeira offprint trafficked immemorial subject micro clinic sprue marx va anaheim eaten1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT502 |
| ✓ | BbhWordSorting169CTSort the following words alphabetically: List: reedy bully multiple edt sell referring lexicographer tit flogging perceptive minor bosporus rancorous forfeit pizza pungent sedition cork1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT1 088 |
| ✓ | BbhWordSorting170CTSort the following words alphabetically: List: thirteenth tinfoil thimble snuff ernie pow celebrity abuilding indiscoverable chigger burgher synchronous yap medlar exorcism newsboy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN999 OUT824 |
Reasoning - BBH word sorting (set 6)
18 passed9 failed |
66.7% |
IN26 694 OUT20 975 | |
| ✓ | BbhWordSorting171CTSort the following words alphabetically: List: saprophytic confident pam stowaway stupor foundling compton1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT440 |
| × | BbhWordSorting172CTSort the following words alphabetically: List: epiphyte swift sail anchor barre dimension faunal integrate eleanor edgy singular patrician orthodoxy parasol pendant buckle concatenate masochist1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("anchor barre buckle concatenate dimension edgy eleanor epiphyte faunal integrate masochist orthodoxy parasol patrician pendant sail singular swift")
Context: BbhWordSorting172CT: expected sorted list, got [anchor barre buckle concatenate dimension edgy eleanor epiphyte faunal integrate masochist orthodoxy parasol pendant patrician sail singular swift] |
Failed | IN993 OUT1 015 |
| ✓ | BbhWordSorting173CTSort the following words alphabetically: List: butterscotch veterinarian kochab molybdate extoller amos farrell academia glance predatory beautiful ford fiducial metzler europium circuitous diatom monomer1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 001 OUT1 022 |
| ✓ | BbhWordSorting174CTSort the following words alphabetically: List: mater rica sigmund dean eosine inebriate gibson vassar mulligatawny formula1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT368 |
| ✓ | BbhWordSorting175CTSort the following words alphabetically: List: shakespearian laudatory1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT120 |
| ✓ | BbhWordSorting176CTSort the following words alphabetically: List: fasciculate judicature presto1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN971 OUT145 |
| ✓ | BbhWordSorting177CTSort the following words alphabetically: List: thereupon wireman crag scrawny electrolytic wife cutover dickson clytemnestra took diocletian lipton inhuman stalk workplace marginal1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT899 |
| × | BbhWordSorting178CTSort the following words alphabetically: List: bruckner admiration technion bandgap heron postprocessor maidenhair fungus allusion gambit accessory proscenium cruise1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("accessory admiration allusion bandgap bruckner cruise fungus gambit heron maidenhair postprocessor proscenium technion")
Context: BbhWordSorting178CT: expected sorted list, got [accessory admiration allusion bandgap bruckner cruise fungus gambit heron maidenhair proscenium postprocessor technion] |
Failed | IN990 OUT705 |
| × | BbhWordSorting179CTSort the following words alphabetically: List: grammarian chivalry polkadot gastronomic ferocity brenda knutson endothelial phenotype irreducible bilinear rockaway derivate eaten cacao scurrilous1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("bilinear brenda cacao chivalry derivate eaten endothelial ferocity gastronomic grammarian irreducible knutson phenotype polkadot rockaway scurrilous")
Context: BbhWordSorting179CT: expected sorted list, got [bilinear brenda cacao chivalry derivate eaten ferocity gastronomic grammarian irreducible knutson phenotype polkadot rockaway scurrilous] |
Failed | IN998 OUT721 |
| ✓ | BbhWordSorting180CTSort the following words alphabetically: List: hyperboloid buckle liquidate alkali nagasaki breach falsetto parmesan mirth1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT509 |
| × | BbhWordSorting181CTSort the following words alphabetically: List: hornblower dissipate amanita canticle annoy besiege straight notre propylene sepia california pasture encephalitis boggle crocodilian dexter snipe amatory dizzy psychiatric1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("amanita amatory annoy besiege boggle california canticle crocodilian dexter dissipate dizzy encephalitis hornblower notre pasture propylene psychiatric sepia snipe straight")
Context: BbhWordSorting181CT: expected sorted list, got [amanita amatory annoy boggle besiege california canticle crocodilian dissipate dexter dizzy encephalitis hornblower notre pasture propylene psychiatric sepia snipe straight] |
Failed | IN1 001 OUT1 336 |
| × | BbhWordSorting182CTSort the following words alphabetically: List: peaceable residue sulfanilamide kingsley eft caliber headboard stamp phillip upholster irishman lobby nary gastrointestinal fragile faulkner capricious ouzo phylum1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("caliber capricious eft faulkner fragile gastrointestinal headboard irishman kingsley lobby nary ouzo peaceable phillip phylum residue stamp sulfanilamide upholster")
Context: BbhWordSorting182CT: expected sorted list, got [caliber capricious eft fragile faulkner gastrointestinal headboard irishman kingsley lobby nary ouzo peaceable phillip phylum residue stamp sulfanilamide upholster] |
Failed | IN1 001 OUT939 |
| ✓ | BbhWordSorting183CTSort the following words alphabetically: List: multiplet catsup hypothalamus scherzo bridesmaid average therewith invocable coddle charm inequity region abbas trumpery memorandum shutout integral dogfish phloem inconvertible1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 005 OUT1 419 |
| ✓ | BbhWordSorting184CTSort the following words alphabetically: List: windowsill appoint biharmonic moustache baneberry wiry dyne pirate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT521 |
| ✓ | BbhWordSorting185CTSort the following words alphabetically: List: zoroaster skimpy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN969 OUT107 |
| × | BbhWordSorting186CTSort the following words alphabetically: List: almost delphic semitic unbeknownst astute deadlock libertarian antic levitate execution sanction scathe storehouse dandelion affable sweeney fortunate leverage horntail apache1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("affable almost antic apache astute dandelion deadlock delphic execution fortunate horntail leverage levitate libertarian sanction scathe semitic storehouse sweeney unbeknownst")
Context: BbhWordSorting186CT: expected sorted list, got [affable almost antic apache astute dandelion deadlock delphic execution fortune horntail leverage levitate libertarian sanction scathe semitic storehouse sweeney unbeknownst] |
Failed | IN999 OUT1 755 |
| × | BbhWordSorting187CTSort the following words alphabetically: List: percent maser roadside we're caching distort phillip emboss epistemology gherkin somewhat spicy injustice winston savoyard delicious defend indicate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("caching defend delicious distort emboss epistemology gherkin indicate injustice maser percent phillip roadside savoyard somewhat spicy we're winston")
Context: BbhWordSorting187CT: expected sorted list, got [caching defend delicious distort emboss epistemology gherkin indicate injustice maser percent phillip roadside savoyard spicy somewhat we're winston] |
Failed | IN994 OUT863 |
| ✓ | BbhWordSorting188CTSort the following words alphabetically: List: anarchic liven shortcut total diatom guelph usage acoustic yarmulke phenomenology neo bureaucracy trammel leftward immovable fabricate syndrome suggestive provide1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT1 580 |
| ✓ | BbhWordSorting189CTSort the following words alphabetically: List: them abdominal address fomalhaut pay optimal hanoverian yang berry tattoo islamabad swathe jordan bounty stearic stigmata involve effusive tornado1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 092 |
| ✓ | BbhWordSorting190CTSort the following words alphabetically: List: maraud downtrodden hurst propane morphine parsonage gadgetry inertial gamin1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN983 OUT485 |
| ✓ | BbhWordSorting191CTSort the following words alphabetically: List: foamflower lawful cayenne chandigarh only excursion declassify equanimity aseptic inclement october debauch kruger groupoid scorch dingy alterate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT1 162 |
| ✓ | BbhWordSorting192CTSort the following words alphabetically: List: acidify antagonism asteria1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT261 |
| × | BbhWordSorting193CTSort the following words alphabetically: List: elton olympic iodinate captious ineligible sherman1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("captious elton ineligible iodinate olympic sherman")
Context: BbhWordSorting193CT: expected sorted list, got [captious elton iodinate ineligible olympic sherman] |
Failed | IN975 OUT222 |
| ✓ | BbhWordSorting194CTSort the following words alphabetically: List: alleviate duopoly mattress gland benelux townsmen buoyant klaxon hardbound tomography felice gunk1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT758 |
| ✓ | BbhWordSorting195CTSort the following words alphabetically: List: solvate nat usable kajar downey vision detest aitken barycentric1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT521 |
| × | BbhWordSorting196CTSort the following words alphabetically: List: griffith common nanosecond loosestrife decoy swabby aerospace denmark yates frill spitz enviable screechy allure sow spermatozoa saute jibe exclusive1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("aerospace allure common decoy denmark enviable exclusive frill griffith jibe loosestrife nanosecond saute screechy sow spermatozoa spitz swabby yates")
Context: BbhWordSorting196CT: expected sorted list, got [aerospace, allure, common, decoy, denmark, enviable, exclusive, frill, griffith, jibe, loosestrife, nanosecond, saute, screechy, sow, spermatozoa, swabby, yates] |
Failed | IN1 003 OUT1 251 |
| ✓ | BbhWordSorting197CTSort the following words alphabetically: List: reverie giantess muddy mast callous bate dnieper prank cortez staunch satisfy dogging moran climb garrison1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN988 OUT759 |
Reasoning - BBH word sorting (set 7)
22 passed4 failed |
84.6% |
IN25 598 OUT14 810 | |
| × | BbhWordSorting198CTSort the following words alphabetically: List: avoid whimsey grassland elegy pitfall error gaillardia hostile caramel conclave stepchild accept carbuncle compressor utopia drib spectra rosa embower1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("accept avoid caramel carbuncle compressor conclave drib elegy embower error gaillardia grassland hostile pitfall rosa spectra stepchild utopia whimsey")
Context: BbhWordSorting198CT: expected sorted list, got [accept, avoid, carbuncle, caramel, compressor, conclave, drib, elegy, embower, error, gaillardia, grassland, hostile, pitfall, rosa, spectra, stepchild, utopia, whimsey] |
Failed | IN995 OUT1 232 |
| ✓ | BbhWordSorting199CTSort the following words alphabetically: List: rowley midshipman saline furl geld locomote exhumation anarchic bstj gradual riddance j liniment pantheist elution profess1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT578 |
| × | BbhWordSorting200CTSort the following words alphabetically: List: hangout drafty supposable sushi proscenium anticonvulsant you'd magnificent scrub embolden hasty pewee carrageen disseminate glamour discovery registrar acuity1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("acuity anticonvulsant carrageen discovery disseminate drafty embolden glamour hangout hasty magnificent pewee proscenium registrar scrub supposable sushi you'd")
Context: BbhWordSorting200CT: expected sorted list, got [acuity, anticonvulsant, carrageen, discovery, dissemination, drafty, embolden, glamour, hangout, hasty, magnificent, pewee, proscenium, registrar, scrub, supposable, sushi, you'd] |
Failed | IN1 001 OUT1 218 |
| ✓ | BbhWordSorting201CTSort the following words alphabetically: List: sweatband murray1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT112 |
| ✓ | BbhWordSorting202CTSort the following words alphabetically: List: resignation dormouse bathos walt amethyst obtuse1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN975 OUT274 |
| × | BbhWordSorting203CTSort the following words alphabetically: List: paginate amicable they've histology extreme indoeuropean sonority tramway turvy coo deerstalker browne henchman pelvis calumny1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("amicable browne calumny coo deerstalker extreme henchman histology indoeuropean paginate pelvis sonority they've tramway turvy")
Context: BbhWordSorting203CT: expected sorted list, got [browne calumny coo deerstalker extreme henchman histology indoeuropean paginate pelvis sonority they've tramway turvy] |
Failed | IN996 OUT668 |
| ✓ | BbhWordSorting204CTSort the following words alphabetically: List: transgressor sidewinder tacky casualty shouldn't courtier leprosy avoidance gibbon merge1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT386 |
| ✓ | BbhWordSorting205CTSort the following words alphabetically: List: intercept tamp geopolitic loci newsmen umber credit atmospheric lunge siren swart chess1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN984 OUT591 |
| ✓ | BbhWordSorting206CTSort the following words alphabetically: List: wink envious scotia planetaria pooh emancipate army1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT362 |
| ✓ | BbhWordSorting207CTSort the following words alphabetically: List: condescend rampant percolate coltish date rochester placid significant1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT481 |
| ✓ | BbhWordSorting208CTSort the following words alphabetically: List: infantrymen knob yaounde aqueous deregulate lysergic gala1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT339 |
| ✓ | BbhWordSorting209CTSort the following words alphabetically: List: promulgate altercate foraminifera sophocles raft wrongdoer syllabus jive cornerstone gossamer courtroom insist dusenberg sal1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT565 |
| ✓ | BbhWordSorting210CTSort the following words alphabetically: List: jacob sleight globule consonant musician1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT126 |
| ✓ | BbhWordSorting211CTSort the following words alphabetically: List: nascent muddy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN967 OUT102 |
| ✓ | BbhWordSorting212CTSort the following words alphabetically: List: automate abner shampoo gurgle none exquisite abramson amity fruitful shorten waterproof1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT377 |
| × | BbhWordSorting213CTSort the following words alphabetically: List: search marcello depreciate unary frame numbly sprout clench agamemnon herbivorous eject zaire lien forum1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("agamemnon clench depreciate eject forum frame herbivorous lien marcello numbly search sprout unary zaire")
Context: BbhWordSorting213CT: expected sorted list, got [agamemnon clench depreciate eject frame forum herbivorous lien marcello numbly search sprout unary zaire] |
Failed | IN990 OUT569 |
| ✓ | BbhWordSorting214CTSort the following words alphabetically: List: saul fresco teen chopstick footwork smalley anaplasmosis palisade orthography storey bumble pilate clue ingot fiesta1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN994 OUT635 |
| ✓ | BbhWordSorting215CTSort the following words alphabetically: List: avalanche chaplin squabble coachman ipsilateral clonic cameroon fortran cram medea referring kennan canal pyridine ussr postpone1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN992 OUT1 578 |
| ✓ | BbhWordSorting216CTSort the following words alphabetically: List: peculiar margaret contravention ingratiate dreg superintendent drapery bizarre sequential1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT523 |
| ✓ | BbhWordSorting217CTSort the following words alphabetically: List: sweatshirt bone ia wagoneer doleful homeobox hindustan convergent1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT407 |
| ✓ | BbhWordSorting218CTSort the following words alphabetically: List: pewter gross gujarati cortland diffusible nomograph stag broadcast galvanometer transition incestuous sketchbook scout larynx1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT838 |
| ✓ | BbhWordSorting219CTSort the following words alphabetically: List: portsmouth impalpable portrait gouda brainy succumb fatuous hoof lisbon erudite malaria enigma isaacson cony stronghold servomechanism1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT764 |
| ✓ | BbhWordSorting220CTSort the following words alphabetically: List: testate orchestra permafrost leasehold shiva1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT248 |
| ✓ | BbhWordSorting221CTSort the following words alphabetically: List: logarithm robertson inanimate uganda micron incurring wilcox lumber sammy souvenir newman profuse maladapt cone butch airlift homeowner1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN993 OUT998 |
| ✓ | BbhWordSorting222CTSort the following words alphabetically: List: chrysalis wallaby1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT118 |
| ✓ | BbhWordSorting223CTSort the following words alphabetically: List: dredge checksum huckster baronial spotlight circumstance eulogy comment felicia dartmouth monochromator neuroanatomic emittance1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN991 OUT721 |
Reasoning - BBH word sorting (set 8)
25 passed1 failed |
96.2% |
IN25 656 OUT17 906 | |
| ✓ | BbhWordSorting224CTSort the following words alphabetically: List: breezy germinal biplane canada nutrient archipelago peloponnese thigh conspiracy constructor accrue dobbin lost hamburger insubstantial malleable ted laramie1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN998 OUT921 |
| ✓ | BbhWordSorting225CTSort the following words alphabetically: List: wagging cabdriver astronomic pivot loch coherent1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT283 |
| ✓ | BbhWordSorting226CTSort the following words alphabetically: List: inability nucleus vague possible invocable janice erg1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN973 OUT471 |
| ✓ | BbhWordSorting227CTSort the following words alphabetically: List: gag grecian ada nobody militia wildcat right abc blend cankerworm hanukkah pierson ron falcon quad flamboyant lobster kruger austere indicate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN996 OUT868 |
| ✓ | BbhWordSorting228CTSort the following words alphabetically: List: babysat curvaceous hugh cutaneous spoke consul stationarity regiment1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN978 OUT467 |
| ✓ | BbhWordSorting229CTSort the following words alphabetically: List: woebegone nimble complementary afternoon immaculate laughlin kind hesitate loire septuagenarian sandia toggle dixie horsepower sierra shuffleboard mechanism1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT753 |
| ✓ | BbhWordSorting230CTSort the following words alphabetically: List: linchpin arrear eavesdropping fasciculate shin postdoctoral maritime ibn henry hermaphrodite sticky jane brookside incorrigible vehicular herodotus1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 003 OUT1 168 |
| ✓ | BbhWordSorting231CTSort the following words alphabetically: List: sinter cortex kangaroo pillage threshold insane mcleod roundabout stipulate marionette trammel incident1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT793 |
| ✓ | BbhWordSorting232CTSort the following words alphabetically: List: soulful dnieper labile lease vehicular1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN974 OUT302 |
| ✓ | BbhWordSorting233CTSort the following words alphabetically: List: cornerstone green brindle prowl nonce ballard scrappy credulous des pitney difluoride toward predilect nostalgic rape mixup horseplay jew curio1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN997 OUT928 |
| ✓ | BbhWordSorting234CTSort the following words alphabetically: List: covenant thence davenport sonorant scoreboard densitometer noisy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN976 OUT473 |
| ✓ | BbhWordSorting235CTSort the following words alphabetically: List: inhabitation bona cicero purport gaussian silver responsible rastus zig exonerate portfolio coastal asset ruanda dusky handlebar1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT656 |
| ✓ | BbhWordSorting236CTSort the following words alphabetically: List: implementor bangui nudge cavalier refractory islamabad contextual abramson waite railway carlisle magistrate silvery picnicking dustbin emacs1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT858 |
| × | BbhWordSorting237CTSort the following words alphabetically: List: strove hendrickson apostasy jacm indonesia vibrate afloat lanthanide grosbeak scavenge quark chattel conner bechtel melancholy ferment1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH Assertion expression evaluated to false: Norm(got) == Norm("afloat apostasy bechtel chattel conner ferment grosbeak hendrickson indonesia jacm lanthanide melancholy quark scavenge strove vibrate")
Context: BbhWordSorting237CT: expected sorted list, got [afloat apostasy bechtel chattel conner ferment grosbeak hendrickson indonesia jacm lanthanide melancholy quark scavenge strive vibrate] |
Failed | IN999 OUT1 005 |
| ✓ | BbhWordSorting238CTSort the following words alphabetically: List: spongy separate gabriel jackson quail hendrickson washy backslide shelter calvert anastomosis variety commando pizzicato sticktight syndicate hollister1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 000 OUT969 |
| ✓ | BbhWordSorting239CTSort the following words alphabetically: List: behold oxalic maybe hew steel termcap pray stiffen dissipate misogyny format dew1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN982 OUT513 |
| ✓ | BbhWordSorting240CTSort the following words alphabetically: List: dove rodriguez sonant monaco strap horticulture claret play paschal bizarre valuate colonist baden deplore1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT700 |
| ✓ | BbhWordSorting241CTSort the following words alphabetically: List: patrolman arabesque novel spearmint arkansan bend brassiere diagnosable ytterbium sheep bedroom yen planeload deterrent fossiliferous alveolar trident fluke curvilinear1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN1 004 OUT1 324 |
| ✓ | BbhWordSorting242CTSort the following words alphabetically: List: papaw perfidious frugal labial feel sonar blunderbuss box dinnertime oresteia1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN986 OUT499 |
| ✓ | BbhWordSorting243CTSort the following words alphabetically: List: haddock colossus sheepskin honeymoon candid pragmatic agrarian people applicate1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN979 OUT344 |
| ✓ | BbhWordSorting244CTSort the following words alphabetically: List: catechism daddy1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN968 OUT105 |
| ✓ | BbhWordSorting245CTSort the following words alphabetically: List: holocene muriel r&d pathetic talon mare staircase bradshaw aperture sigh1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN981 OUT740 |
| ✓ | BbhWordSorting246CTSort the following words alphabetically: List: bust midshipmen musicale urinary chalk rocket resiny horror tangential sapient dumb serf pathway urea goucher fatty masonry dentistry cowboy roadrunner1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN995 OUT1 400 |
| ✓ | BbhWordSorting247CTSort the following words alphabetically: List: sanhedrin scratchy helical beau venezuela awash bessie extricable indoeuropean vice pendulum cream animism1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN989 OUT699 |
| ✓ | BbhWordSorting248CTSort the following words alphabetically: List: him correspond seashore herpes1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN970 OUT249 |
| ✓ | BbhWordSorting249CTSort the following words alphabetically: List: worcestershire bonito fritter dreamboat haggard nose whodunit1 AI query
0 tool callscategory:Reasoninglevel:VeryHardsource:BBH |
Passed | IN980 OUT418 |